Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
  claude BotandClaude Opus 5.5
|
aff7cf8839 | ||
|
claude Bot
|
389a679c9f | ||
|
claude BotandClaude Opus 5.5
|
8b5ee73ee1 | ||
|
claude Bot
|
e901de442c | ||
|
claude Bot
|
2f84bc8deb | ||
|
claude Bot
|
524f5dc0fd | ||
 cedricandClaude Opus 5.5
|
4f4cb3e02b | ||
|
cedricandClaude Opus 5.5
|
42f6137391 | ||
|
cedricandClaude Opus 5.5
|
3504263992 | ||
|
claude Bot
|
3466a29692 |
No files matched your search
+8
-1
@@ -9,6 +9,9 @@ AUTH_MODE=local
|
||||
# local mode: user and password (empty = no authentication)
|
||||
AUTH_USER=
|
||||
AUTH_PASS=
|
||||
# local mode: optional read-only account (can search and export, cannot change tags, sources or purge)
|
||||
AUTH_VIEWER_USER=
|
||||
AUTH_VIEWER_PASS=
|
||||
# local mode: PNG logo shown on the login page, path inside the container (empty = no logo).
|
||||
# Mount the file in docker-compose.yml, e.g. ./logo.png:/config/logo.png:ro, then LOGIN_LOGO=/config/logo.png
|
||||
LOGIN_LOGO=
|
||||
@@ -28,12 +31,16 @@ OIDC_CLIENT_SECRET=
|
||||
OIDC_REDIRECT_URL=https://logs.example.org/auth/callback
|
||||
# Requested scopes (openid is always added)
|
||||
OIDC_SCOPES=openid profile email
|
||||
# oidc mode: only members of this group are admins, the others are read-only (empty = everyone is admin).
|
||||
# The groups come from the ID token claim OIDC_GROUPS_CLAIM (default groups)
|
||||
OIDC_ADMIN_GROUP=
|
||||
OIDC_GROUPS_CLAIM=groups
|
||||
# Reverse DNS: show host names instead of IP addresses (on/off)
|
||||
RDNS=on
|
||||
# DNS server used for reverse lookups (e.g. your router: 192.168.1.1). Empty = system resolver
|
||||
DNS_SERVER=
|
||||
# Allow "Delete all logs" in Settings (true/false)
|
||||
ALLOW_PURGE=true
|
||||
ALLOW_PURGE=false
|
||||
# Maximum number of rows in a CSV export
|
||||
EXPORT_MAX=100000
|
||||
# Collect the logs of the Docker containers of this machine (on/off)
|
||||
|
||||
+53
-15
@@ -18,8 +18,10 @@ devices ──514 udp/tcp──▶ logstream (Go) ──HTTP batches──▶ Vi
|
||||
|
||||
|
||||
- **Ingestion** : les messages syslog (UDP/TCP) et les logs des conteneurs Docker passent tous
|
||||
par `sink()` (résolution DNS inverse des hôtes donnés par leur IP), puis par la file du
|
||||
`Store`, qui les envoie par lots à VictoriaLogs.
|
||||
par `sink()` (résolution DNS inverse des hôtes donnés par leur IP, sans bloquer la
|
||||
réception), puis par la file du `Store`, qui les envoie par lots à VictoriaLogs. Quand
|
||||
VictoriaLogs est injoignable, les lots sont gardés sur disque (`/data/spool`, jusqu'à
|
||||
`SPOOL_MAX_MB`) et renvoyés, les plus anciens d'abord, dès son retour.
|
||||
- **Direct** : `sink()` publie aussi chaque message dans le `Hub`, qui le diffuse aux
|
||||
navigateurs en SSE.
|
||||
- **Recherche** : l'API HTTP traduit les filtres de l'interface en requêtes LogsQL envoyées à
|
||||
@@ -100,6 +102,16 @@ restant. Les largeurs sont mémorisées par le navigateur (**Paramètres > Inter
|
||||
Réinitialiser les colonnes** les rétablit toutes). Sur téléphone, la liste garde sa
|
||||
présentation sur deux lignes, sans colonnes.
|
||||
|
||||
## Modes Flux et Période
|
||||
|
||||
Le premier contrôle de la barre de filtres bascule entre deux modes d'affichage :
|
||||
|
||||
- **Flux** : les derniers logs sur une durée glissante (5 min à 30 jours, ou tout), avec le direct.
|
||||
- **Période** : les logs entre une date de début et une date de fin, saisies dans le fuseau choisi
|
||||
dans les Paramètres. ◀ et ▶ passent à la période précédente ou suivante de même durée, la loupe
|
||||
la double autour de son milieu. Le direct se met en pause ; le mode et la période sont conservés
|
||||
au rechargement.
|
||||
|
||||
## Frise
|
||||
|
||||
La frise au-dessus de la liste montre le volume de logs par intervalle, compté selon l'**heure
|
||||
@@ -107,10 +119,9 @@ de réception** sur l'horloge du serveur (elle correspond donc aux heures affich
|
||||
lignes).
|
||||
|
||||
- Au survol d'un intervalle : ses bornes, son total et le détail par sévérité.
|
||||
- Un clic sur une barre zoome sur cet intervalle ; un glisser sur plusieurs barres zoome sur la
|
||||
sélection. La plage de temps affiche alors la période zoomée (« × Annuler le zoom » ou le
|
||||
choix d'une autre plage en sort) ; la liste, les compteurs et l'export CSV suivent le zoom,
|
||||
et le direct se met en pause.
|
||||
- Un clic sur une barre affiche cet intervalle en mode Période ; un glisser sur plusieurs barres
|
||||
affiche la sélection. « × Revenir au flux » repasse en mode Flux. La liste, les compteurs et
|
||||
l'export CSV suivent la période.
|
||||
- En direct, le dernier intervalle grandit à l'arrivée des messages, et la frise se recharge à
|
||||
chaque nouvel intervalle. Rien n'est rafraîchi tant que l'onglet du navigateur est masqué ;
|
||||
la frise se met à jour dès qu'il redevient visible.
|
||||
@@ -153,8 +164,10 @@ colorés et exportés comme les messages syslog :
|
||||
quand les conteneurs sont nombreux. Les nouveaux conteneurs sont suivis automatiquement, sauf
|
||||
si cette option est désactivée. Les choix sont enregistrés par service compose (ou nom de
|
||||
conteneur) dans `/data/docker.json` : ils survivent aux recréations.
|
||||
- Logstream mémorise la position lue dans chaque conteneur (`/data/docker-state.json`) : après
|
||||
un redémarrage, il reprend sans perdre ni dupliquer de lignes. Un conteneur vu pour la
|
||||
- Logstream mémorise la position de la dernière ligne stockée pour chaque conteneur
|
||||
(`/data/docker-state.json`) : après un redémarrage, il reprend sans perdre de lignes. La
|
||||
position n'avance qu'une fois la ligne dans VictoriaLogs ou dans le tampon disque, et une file
|
||||
pleine ralentit la lecture au lieu de perdre des lignes. Un conteneur vu pour la
|
||||
première fois est lu à partir de `DOCKER_BACKFILL` en arrière (1 heure par défaut).
|
||||
- Logstream lui-même et le proxy ci-dessous ne sont jamais collectés ; ajoutez l'étiquette
|
||||
`logstream.exclude=true` à tout autre conteneur pour l'exclure définitivement.
|
||||
@@ -257,11 +270,15 @@ couleur, est mémorisé par navigateur.
|
||||
européen de polices respectueux de la vie privée ; sans accès à internet, la police du
|
||||
système est utilisée. Les ligatures sont désactivées pour que `->` ou `!=` s'affichent tels
|
||||
quels. Le réglage le plus dense est Très petite + Compacte + Inconsolata Condensed.
|
||||
- **Données** : « Supprimer tous les logs » efface définitivement tous les logs stockés (il faut
|
||||
- **Données** : chiffres de la base lus dans VictoriaLogs (actualisés au plus toutes les 30 s) :
|
||||
lignes stockées, taille sur disque (index compris), taille brute et taux de compression,
|
||||
période couverte avec la rétention, lignes des dernières 24 h et de la dernière heure, hôtes
|
||||
et applications distincts, espace disque libre. Les tailles sont en Ko/Mo/Go (KB/MB/GB en
|
||||
anglais). « Supprimer tous les logs » efface définitivement tous les logs stockés (il faut
|
||||
taper `PURGE` pour confirmer). Les tags et les paramètres sont conservés. VictoriaLogs doit
|
||||
être lancé avec `-delete.enable` (déjà présent dans `docker-compose.yml`) ; mettez
|
||||
`ALLOW_PURGE=false` pour désactiver la fonction. Toute personne qui peut ouvrir l'interface
|
||||
peut purger : activez l'[authentification](#authentification) si l'interface est accessible
|
||||
être lancé avec `-delete.enable` (déjà présent dans `docker-compose.yml`). La fonction est
|
||||
désactivée par défaut : mettez `ALLOW_PURGE=true` pour l'autoriser. Tout administrateur peut
|
||||
alors purger : activez l'[authentification](#authentification) si l'interface est accessible
|
||||
à d'autres.
|
||||
|
||||
## Authentification
|
||||
@@ -270,6 +287,7 @@ couleur, est mémorisé par navigateur.
|
||||
|
||||
- **`local`** (par défaut) : une page de connexion avec le compte `AUTH_USER` / `AUTH_PASS` ; laissez-les
|
||||
vides pour n'avoir aucune authentification (par exemple derrière un reverse proxy qui contrôle déjà).
|
||||
L'interface affiche alors un bandeau d'avertissement, que l'on peut fermer.
|
||||
- **`oidc`** : connexion par un fournisseur OpenID Connect (Keycloak, Authentik, Authelia, Zitadel…),
|
||||
flux « authorization code » avec PKCE.
|
||||
|
||||
@@ -280,6 +298,10 @@ déconnexion (en haut à droite) y met fin. Les échecs de connexion sont écrit
|
||||
l'adresse du client (`auth: failed login for "bob" from 192.0.2.7`). Les scripts peuvent toujours
|
||||
appeler l'API avec des identifiants HTTP Basic (`curl -u utilisateur:motdepasse`).
|
||||
|
||||
Un **compte en lecture seule** optionnel, `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS`, peut chercher,
|
||||
suivre le direct et exporter, mais pas modifier les tags, les sources ni purger : ces réglages
|
||||
sont grisés dans son interface et l'API répond `403`.
|
||||
|
||||
Pour afficher votre logo sur la page de connexion, montez un PNG dans le conteneur et indiquez
|
||||
son chemin dans `LOGIN_LOGO` :
|
||||
|
||||
@@ -320,6 +342,16 @@ l'application). Les connexions sont écrites dans les logs de logstream (`oidc:
|
||||
Avec une URL de retour en `https`, les cookies ne sont envoyés qu'en HTTPS : logstream doit être
|
||||
joint à travers un reverse proxy TLS.
|
||||
|
||||
Pour donner un accès en lecture seule à certains utilisateurs, définissez `OIDC_ADMIN_GROUP` (par
|
||||
exemple `logstream-admins`) : seuls ses membres sont administrateurs, les autres sont en lecture
|
||||
seule. Les groupes sont lus dans la revendication `groups` du jeton d'identité
|
||||
(`OIDC_GROUPS_CLAIM` pour en utiliser une autre) ; dans Keycloak, ajoutez au client un mapper
|
||||
« Group Membership » (le `/` initial est ignoré).
|
||||
|
||||
Quel que soit le mode, chaque réponse porte des en-têtes de sécurité (Content-Security-Policy,
|
||||
X-Frame-Options…), et l'API refuse les modifications envoyées depuis un autre site (requêtes
|
||||
intersites).
|
||||
|
||||
## Noms d'hôtes (DNS inverse)
|
||||
|
||||
Quand un équipement envoie son adresse IP comme nom d'hôte (ou pas de nom d'hôte du tout),
|
||||
@@ -342,15 +374,20 @@ résolutions.
|
||||
| `RETENTION` | `30d` | durée de conservation des logs dans VictoriaLogs |
|
||||
| `AUTH_MODE` | `local` | `local` (page de connexion) ou `oidc`, voir [Authentification](#authentification) |
|
||||
| `AUTH_USER` / `AUTH_PASS` | vide | compte de la page de connexion (mode `local`) ; vide = pas d'authentification |
|
||||
| `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS` | vide | compte optionnel en lecture seule (mode `local`) |
|
||||
| `LOGIN_LOGO` | vide | PNG affiché sur la page de connexion, chemin dans le conteneur (mode `local`) |
|
||||
| `SESSION_TTL` | `12h` | durée de la session (les deux modes ; `OIDC_SESSION_TTL` fonctionne toujours) |
|
||||
| `OIDC_ISSUER` | vide | URL de l'issuer du fournisseur OpenID Connect (mode `oidc`) |
|
||||
| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | vide | client déclaré dans le fournisseur |
|
||||
| `OIDC_REDIRECT_URL` | vide | URL de retour de logstream, ex. `https://logs.example.org/auth/callback` |
|
||||
| `OIDC_SCOPES` | `openid profile email` | scopes demandés |
|
||||
| `OIDC_ADMIN_GROUP` | vide | seuls les membres de ce groupe sont administrateurs, les autres en lecture seule (vide = tout le monde est administrateur) |
|
||||
| `OIDC_GROUPS_CLAIM` | `groups` | revendication du jeton d'identité qui liste les groupes |
|
||||
| `RDNS` | `on` | résoudre les hôtes donnés par leur IP en noms DNS |
|
||||
| `DNS_SERVER` | vide | serveur DNS pour les résolutions inverses (`ip` ou `ip:port`) |
|
||||
| `ALLOW_PURGE` | `true` | autoriser « Supprimer tous les logs » dans les Paramètres |
|
||||
| `ALLOW_PURGE` | `false` | autoriser « Supprimer tous les logs » dans les Paramètres |
|
||||
| `SYSLOG_TCP_MAX_CONNS` | `512` | connexions syslog TCP ouvertes en même temps ; au-delà, elles sont refusées |
|
||||
| `SYSLOG_TCP_IDLE` | `30m` | une connexion syslog TCP silencieuse pendant cette durée est fermée (les émetteurs se reconnectent) |
|
||||
| `EXPORT_MAX` | `100000` | nombre maximal de lignes dans un export CSV |
|
||||
| `PRESETS_FILE` | `/data/presets.json` | fichier des préréglages de tags ; liste intégrée s'il est absent (voir [docs/presets.fr.md](docs/presets.fr.md)) |
|
||||
| `DOCKER_LOGS` | `on` dans compose | collecter les logs des conteneurs Docker locaux |
|
||||
@@ -361,13 +398,14 @@ résolutions.
|
||||
| `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte |
|
||||
| `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) |
|
||||
| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion |
|
||||
| `SPOOL_MAX_MB` | `1024` | taille du tampon disque des lots refusés par VictoriaLogs (`0` = pas de tampon : 15 s de tentatives, puis perte) |
|
||||
|
||||
## Débogage
|
||||
|
||||
- `docker compose logs -f logstream` : erreurs de réception et erreurs d'envoi vers
|
||||
VictoriaLogs.
|
||||
- La barre du bas affiche les compteurs reçus / stockés / perdus et la dernière erreur de
|
||||
stockage.
|
||||
- La barre du bas affiche les compteurs reçus / stockés / perdus, les messages en attente dans
|
||||
le tampon disque et la dernière erreur de stockage.
|
||||
- <http://localhost:9428/select/vmui> : l'interface de VictoriaLogs, pour essayer des requêtes
|
||||
LogsQL.
|
||||
- API :
|
||||
|
||||
@@ -17,7 +17,9 @@ devices ──514 udp/tcp──▶ logstream (Go) ──HTTP batches──▶ Vi
|
||||
|
||||
|
||||
- **Ingestion**: syslog (UDP/TCP) and Docker container logs both go through `sink()` (reverse DNS
|
||||
on IP hosts), then the `Store` queue, which sends them in batches to VictoriaLogs.
|
||||
on IP hosts, without holding up the listeners), then the `Store` queue, which sends them in
|
||||
batches to VictoriaLogs. When VictoriaLogs is unreachable, batches are kept on disk
|
||||
(`/data/spool`, up to `SPOOL_MAX_MB`) and sent again, oldest first, once it is back.
|
||||
- **Live view**: `sink()` also publishes each message to the `Hub`, which streams it to the
|
||||
browsers over SSE.
|
||||
- **Search**: the HTTP API turns the UI filters into LogsQL queries sent to VictoriaLogs.
|
||||
@@ -90,15 +92,25 @@ width; the message takes the remaining space. Widths are remembered by the brows
|
||||
(**Settings > Interface > Reset column widths** restores them all). On phones the list keeps its
|
||||
two-line layout without columns.
|
||||
|
||||
## Stream and Time range modes
|
||||
|
||||
The first control of the filter bar switches between two display modes:
|
||||
|
||||
- **Stream**: the latest logs over a sliding duration (5 min to 30 days, or all), with the live
|
||||
view.
|
||||
- **Time range**: the logs between a start and an end date, typed in the time zone chosen in
|
||||
Settings. ◀ and ▶ move to the previous or next range of the same length, the magnifier doubles
|
||||
it around its middle. The live view pauses; the mode and the range are kept across reloads.
|
||||
|
||||
## Timeline
|
||||
|
||||
The timeline above the list shows the volume of logs per interval, counted by **reception
|
||||
time** on the server clock (so it matches the times shown in the rows).
|
||||
|
||||
- Hover an interval: its bounds, total and detail per severity.
|
||||
- Click a bar to zoom on that interval, or drag across several bars to zoom on the selection.
|
||||
The time range then shows the zoomed period ("× Reset zoom" or any other range leaves it);
|
||||
the list, the counters and the CSV export follow the zoom, and the live view pauses.
|
||||
- Click a bar to show that interval in Time range mode, or drag across several bars to show the
|
||||
selection; "× Back to stream" returns to Stream mode. The list, the counters and the CSV export
|
||||
follow the range.
|
||||
- In live mode the last interval grows as messages arrive, and the timeline reloads at each new
|
||||
interval. Nothing is refreshed while the browser tab is hidden; it catches up when shown again.
|
||||
|
||||
@@ -137,8 +149,10 @@ colored and exported like syslog messages:
|
||||
to the labels shown) help with many containers. New containers are followed automatically
|
||||
unless that option is turned off. Choices are saved per compose service (or container name)
|
||||
in `/data/docker.json`, so they survive re-creations.
|
||||
- Logstream remembers the position read in each container (`/data/docker-state.json`): after a
|
||||
restart it resumes without losing or duplicating lines. A container seen for the first time
|
||||
- Logstream remembers the position of the last line stored for each container
|
||||
(`/data/docker-state.json`): after a restart it resumes without losing lines. The position
|
||||
only moves once a line is in VictoriaLogs or in the disk buffer, and a full queue slows the
|
||||
reading down instead of dropping lines. A container seen for the first time
|
||||
is read from `DOCKER_BACKFILL` ago (1 hour by default).
|
||||
- Logstream itself and the proxy below are never collected; add the label
|
||||
`logstream.exclude=true` to any other container to exclude it for good.
|
||||
@@ -233,10 +247,13 @@ remembered per browser.
|
||||
[Bunny Fonts](https://fonts.bunny.net), a privacy-friendly European font service; without
|
||||
internet access, the system font is used. Ligatures are disabled so `->` or `!=` show as
|
||||
typed. The densest setting is Tiny + Compact + Inconsolata Condensed.
|
||||
- **Data**: "Delete all logs" permanently erases every stored log (you must type
|
||||
- **Data**: database figures read from VictoriaLogs (refreshed at most every 30 s): stored
|
||||
lines, size on disk (index included), raw size and compression ratio, period covered with
|
||||
the retention, lines of the last 24 h and last hour, distinct hosts and apps, free disk
|
||||
space. Sizes use KB/MB/GB (Ko/Mo/Go in French). "Delete all logs" permanently erases every stored log (you must type
|
||||
`PURGE` to confirm). Tags and settings are kept. VictoriaLogs needs `-delete.enable`
|
||||
(already set in `docker-compose.yml`); set `ALLOW_PURGE=false` to disable the feature.
|
||||
Anyone who can open the UI can purge: turn on [authentication](#authentication) if the UI
|
||||
(already set in `docker-compose.yml`). The feature is off by default: set `ALLOW_PURGE=true`
|
||||
to allow it. Any admin can then purge: turn on [authentication](#authentication) if the UI
|
||||
is reachable by others.
|
||||
|
||||
## Authentication
|
||||
@@ -244,7 +261,8 @@ remembered per browser.
|
||||
`AUTH_MODE` picks how the UI and the API are protected (`/healthz` always stays open):
|
||||
|
||||
- **`local`** (default): a login page with the account `AUTH_USER` / `AUTH_PASS`; leave them
|
||||
empty to have no authentication (for instance behind a reverse proxy that already checks).
|
||||
empty to have no authentication (for instance behind a reverse proxy that already checks). The
|
||||
UI then shows a warning banner, which can be closed.
|
||||
- **`oidc`**: login through an OpenID Connect provider (Keycloak, Authentik, Authelia, Zitadel…),
|
||||
authorization code flow with PKCE.
|
||||
|
||||
@@ -254,6 +272,10 @@ ends when `AUTH_USER` or `AUTH_PASS` changes; the log out button (top right) end
|
||||
are written in the logs with the client address (`auth: failed login for "bob" from 192.0.2.7`).
|
||||
Scripts can still call the API with HTTP Basic credentials (`curl -u user:pass`).
|
||||
|
||||
An optional **read-only account**, `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS`, can search, follow
|
||||
the live view and export, but cannot change tags, sources or purge: those settings are greyed
|
||||
out in its UI and the API answers `403`.
|
||||
|
||||
To show your logo on the login page, mount a PNG in the container and point `LOGIN_LOGO` to it:
|
||||
|
||||
```yaml
|
||||
@@ -291,6 +313,14 @@ Every user the provider accepts for this client can log in: restrict access in t
|
||||
in the logstream logs (`oidc: alice logged in`). With an `https` redirect URL, the cookies are
|
||||
only sent over HTTPS: logstream must be reached through a TLS reverse proxy.
|
||||
|
||||
To give read-only access to some users, set `OIDC_ADMIN_GROUP` (for instance
|
||||
`logstream-admins`): only its members are admins, the others are read-only. The groups are read
|
||||
from the `groups` claim of the ID token (`OIDC_GROUPS_CLAIM` to use another one); in Keycloak,
|
||||
add a "Group Membership" mapper to the client (a leading `/` is ignored).
|
||||
|
||||
Whatever the mode, every answer carries security headers (Content-Security-Policy,
|
||||
X-Frame-Options…), and the API refuses changes sent from another site (cross-site requests).
|
||||
|
||||
## Host names (reverse DNS)
|
||||
|
||||
When a device sends its IP address as host name (or no host name at all), Logstream looks up
|
||||
@@ -311,15 +341,20 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
|
||||
| `RETENTION` | `30d` | how long VictoriaLogs keeps logs |
|
||||
| `AUTH_MODE` | `local` | `local` (login page) or `oidc`, see [Authentication](#authentication) |
|
||||
| `AUTH_USER` / `AUTH_PASS` | empty | account of the login page (`local` mode); empty = no authentication |
|
||||
| `AUTH_VIEWER_USER` / `AUTH_VIEWER_PASS` | empty | optional read-only account (`local` mode) |
|
||||
| `LOGIN_LOGO` | empty | PNG shown on the login page, path inside the container (`local` mode) |
|
||||
| `SESSION_TTL` | `12h` | session lifetime (both modes; `OIDC_SESSION_TTL` still works) |
|
||||
| `OIDC_ISSUER` | empty | issuer URL of the OpenID Connect provider (`oidc` mode) |
|
||||
| `OIDC_CLIENT_ID` / `OIDC_CLIENT_SECRET` | empty | client registered in the provider |
|
||||
| `OIDC_REDIRECT_URL` | empty | callback URL of logstream, e.g. `https://logs.example.org/auth/callback` |
|
||||
| `OIDC_SCOPES` | `openid profile email` | requested scopes |
|
||||
| `OIDC_ADMIN_GROUP` | empty | only members of this group are admins, the others read-only (empty = everyone is admin) |
|
||||
| `OIDC_GROUPS_CLAIM` | `groups` | ID token claim that lists the groups |
|
||||
| `RDNS` | `on` | resolve IP hosts to DNS names |
|
||||
| `DNS_SERVER` | empty | DNS server for reverse lookups (`ip` or `ip:port`) |
|
||||
| `ALLOW_PURGE` | `true` | allow "Delete all logs" in Settings |
|
||||
| `ALLOW_PURGE` | `false` | allow "Delete all logs" in Settings |
|
||||
| `SYSLOG_TCP_MAX_CONNS` | `512` | syslog TCP connections open at once; more are refused |
|
||||
| `SYSLOG_TCP_IDLE` | `30m` | a syslog TCP connection silent this long is closed (senders reconnect) |
|
||||
| `EXPORT_MAX` | `100000` | maximum number of rows in a CSV export |
|
||||
| `PRESETS_FILE` | `/data/presets.json` | color tag presets file; the built-in list when missing (see [docs/presets.md](docs/presets.md)) |
|
||||
| `DOCKER_LOGS` | `on` in compose | collect the logs of the local Docker containers |
|
||||
@@ -330,11 +365,13 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
|
||||
| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted |
|
||||
| `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) |
|
||||
| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning |
|
||||
| `SPOOL_MAX_MB` | `1024` | disk buffer size for batches VictoriaLogs could not take (`0` = no buffer: retried for 15 s, then dropped) |
|
||||
|
||||
## Debugging
|
||||
|
||||
- `docker compose logs -f logstream`: receive errors and errors sending to VictoriaLogs.
|
||||
- The bottom bar shows received / stored / dropped counters and the last storage error.
|
||||
- The bottom bar shows received / stored / dropped counters, the messages waiting in the disk
|
||||
buffer, and the last storage error.
|
||||
- <http://localhost:9428/select/vmui>: VictoriaLogs' own UI to try LogsQL queries.
|
||||
- API:
|
||||
```bash
|
||||
|
||||
@@ -23,6 +23,7 @@ type API struct {
|
||||
docker *DockerManager // nil when DOCKER_LOGS is off
|
||||
syslog *SyslogServer
|
||||
host *HostLogs
|
||||
dbstats dbStatsCache
|
||||
}
|
||||
|
||||
func (a *API) Routes(mux *http.ServeMux) {
|
||||
@@ -32,6 +33,7 @@ func (a *API) Routes(mux *http.ServeMux) {
|
||||
mux.HandleFunc("GET /api/facets", a.facets)
|
||||
mux.HandleFunc("GET /api/stream", a.stream)
|
||||
mux.HandleFunc("GET /api/stats", a.stats)
|
||||
mux.HandleFunc("GET /api/dbstats", a.dbStats)
|
||||
mux.HandleFunc("GET /api/tags", a.listTags)
|
||||
mux.HandleFunc("POST /api/tags", a.createTag)
|
||||
mux.HandleFunc("POST /api/tags/reset", a.resetTags)
|
||||
@@ -314,6 +316,17 @@ func (a *API) stats(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, s)
|
||||
}
|
||||
|
||||
// GET /api/dbstats[?refresh=1]: size and content of the VictoriaLogs base
|
||||
// (Settings > Data), cached for 30 s.
|
||||
func (a *API) dbStats(w http.ResponseWriter, r *http.Request) {
|
||||
st, err := a.dbstats.get(r.Context(), a.store, r.URL.Query().Get("refresh") == "1")
|
||||
if err != nil {
|
||||
writeErr(w, http.StatusBadGateway, err)
|
||||
return
|
||||
}
|
||||
writeJSON(w, http.StatusOK, st)
|
||||
}
|
||||
|
||||
func (a *API) listTags(w http.ResponseWriter, r *http.Request) {
|
||||
writeJSON(w, http.StatusOK, a.tags.List())
|
||||
}
|
||||
|
||||
@@ -42,6 +42,10 @@ const (
|
||||
type authConfig struct {
|
||||
mode string
|
||||
user, pass string // local mode
|
||||
viewerUser string // local mode: optional read-only account
|
||||
viewerPass string
|
||||
adminGroup string // oidc: only members of this group are admins (empty: everyone)
|
||||
groupsClaim string // oidc: ID token claim listing the groups
|
||||
issuer string
|
||||
clientID string
|
||||
clientSecret string
|
||||
@@ -124,6 +128,9 @@ func newOIDC(c authConfig) (*OIDC, error) {
|
||||
if !strings.Contains(" "+c.scopes+" ", " openid ") {
|
||||
c.scopes = "openid " + c.scopes
|
||||
}
|
||||
if c.groupsClaim == "" {
|
||||
c.groupsClaim = "groups"
|
||||
}
|
||||
return &OIDC{
|
||||
cfg: c,
|
||||
callback: ru.Path,
|
||||
@@ -157,8 +164,9 @@ func sessionKey(dir string) []byte {
|
||||
}
|
||||
|
||||
type session struct {
|
||||
User string `json:"u"`
|
||||
Exp int64 `json:"e"`
|
||||
User string `json:"u"`
|
||||
Exp int64 `json:"e"`
|
||||
Viewer bool `json:"v,omitempty"` // read-only user
|
||||
}
|
||||
|
||||
// writeAuthRequired answers API calls without a session; the UI turns it into a reload
|
||||
@@ -191,9 +199,12 @@ func (o *OIDC) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
var s session
|
||||
if c, err := r.Cookie(sessionCookie); err == nil && verifyCookie(o.key, c.Value, &s) && time.Now().Unix() < s.Exp {
|
||||
if r.URL.Path == "/auth/me" {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"mode": "oidc", "user": s.User})
|
||||
writeJSON(w, http.StatusOK, map[string]string{"mode": "oidc", "user": s.User, "role": roleName(s.Viewer)})
|
||||
return
|
||||
}
|
||||
if s.Viewer {
|
||||
r = asViewer(r)
|
||||
}
|
||||
o.next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
@@ -257,16 +268,16 @@ func (o *OIDC) handleCallback(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
http.SetCookie(w, &http.Cookie{Name: loginCookie + state, Path: "/", MaxAge: -1, HttpOnly: true, Secure: o.secure})
|
||||
|
||||
user, err := o.exchange(r, q.Get("code"), ls)
|
||||
user, viewer, err := o.exchange(r, q.Get("code"), ls)
|
||||
if err != nil {
|
||||
log.Printf("oidc: login failed: %v", err)
|
||||
http.Error(w, "login failed, see the LogStream logs", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
log.Printf("oidc: %s logged in", user)
|
||||
log.Printf("oidc: %s logged in (%s)", user, roleName(viewer))
|
||||
http.SetCookie(w, &http.Cookie{
|
||||
Name: sessionCookie,
|
||||
Value: signCookie(o.key, session{User: user, Exp: time.Now().Add(o.cfg.sessionTTL).Unix()}),
|
||||
Value: signCookie(o.key, session{User: user, Exp: time.Now().Add(o.cfg.sessionTTL).Unix(), Viewer: viewer}),
|
||||
Path: "/",
|
||||
MaxAge: int(o.cfg.sessionTTL.Seconds()),
|
||||
HttpOnly: true,
|
||||
@@ -286,14 +297,15 @@ func (o *OIDC) handleLogout(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, "/", http.StatusFound)
|
||||
}
|
||||
|
||||
// exchange trades the code for tokens and returns the user name from the verified ID token.
|
||||
func (o *OIDC) exchange(r *http.Request, code string, ls loginState) (string, error) {
|
||||
// exchange trades the code for tokens and returns the user name from the verified ID
|
||||
// token, and whether the user is read-only (not in OIDC_ADMIN_GROUP).
|
||||
func (o *OIDC) exchange(r *http.Request, code string, ls loginState) (string, bool, error) {
|
||||
if code == "" {
|
||||
return "", errors.New("no code in the callback")
|
||||
return "", false, errors.New("no code in the callback")
|
||||
}
|
||||
meta, err := o.discover()
|
||||
if err != nil {
|
||||
return "", err
|
||||
return "", false, err
|
||||
}
|
||||
form := url.Values{
|
||||
"grant_type": {"authorization_code"},
|
||||
@@ -309,7 +321,7 @@ func (o *OIDC) exchange(r *http.Request, code string, ls loginState) (string, er
|
||||
}
|
||||
req, err := http.NewRequestWithContext(r.Context(), http.MethodPost, meta.TokenEndpoint, strings.NewReader(form.Encode()))
|
||||
if err != nil {
|
||||
return "", err
|
||||
return "", false, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
|
||||
req.Header.Set("Accept", "application/json")
|
||||
@@ -318,29 +330,53 @@ func (o *OIDC) exchange(r *http.Request, code string, ls loginState) (string, er
|
||||
}
|
||||
res, err := o.client.Do(req)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("token endpoint: %w", err)
|
||||
return "", false, fmt.Errorf("token endpoint: %w", err)
|
||||
}
|
||||
defer res.Body.Close()
|
||||
body, _ := io.ReadAll(io.LimitReader(res.Body, 1<<20))
|
||||
if res.StatusCode != http.StatusOK {
|
||||
return "", fmt.Errorf("token endpoint: %s: %s", res.Status, bytes.TrimSpace(body))
|
||||
return "", false, fmt.Errorf("token endpoint: %s: %s", res.Status, bytes.TrimSpace(body))
|
||||
}
|
||||
var tok struct {
|
||||
IDToken string `json:"id_token"`
|
||||
}
|
||||
if err := json.Unmarshal(body, &tok); err != nil || tok.IDToken == "" {
|
||||
return "", errors.New("token endpoint: no id_token in the response")
|
||||
return "", false, errors.New("token endpoint: no id_token in the response")
|
||||
}
|
||||
claims, err := o.verifyIDToken(tok.IDToken, ls.Nonce)
|
||||
if err != nil {
|
||||
return "", err
|
||||
return "", false, err
|
||||
}
|
||||
viewer := o.cfg.adminGroup != "" && !hasGroup(claims[o.cfg.groupsClaim], o.cfg.adminGroup)
|
||||
for _, k := range []string{"preferred_username", "email", "name", "sub"} {
|
||||
if v, _ := claims[k].(string); v != "" {
|
||||
return v, nil
|
||||
return v, viewer, nil
|
||||
}
|
||||
}
|
||||
return "", errors.New("id_token: no sub")
|
||||
return "", false, errors.New("id_token: no sub")
|
||||
}
|
||||
|
||||
// hasGroup tells whether the groups claim (a list, or a single string) holds
|
||||
// group; a leading "/" (Keycloak group paths) is ignored.
|
||||
func hasGroup(claim any, group string) bool {
|
||||
group = strings.TrimPrefix(group, "/")
|
||||
var groups []string
|
||||
switch v := claim.(type) {
|
||||
case string:
|
||||
groups = strings.Fields(strings.ReplaceAll(v, ",", " "))
|
||||
case []any:
|
||||
for _, g := range v {
|
||||
if s, ok := g.(string); ok {
|
||||
groups = append(groups, s)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, g := range groups {
|
||||
if strings.TrimPrefix(g, "/") == group {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// verifyIDToken checks the signature (keys from jwks_uri) and the claims of an ID token.
|
||||
|
||||
+38
-16
@@ -23,6 +23,8 @@ var loginFailDelay = time.Second // slows down password guessing
|
||||
|
||||
type Local struct {
|
||||
user, pass string
|
||||
viewerUser string // optional read-only account
|
||||
viewerPass string
|
||||
ttl time.Duration
|
||||
logo string // LOGIN_LOGO, served at /auth/logo
|
||||
key []byte
|
||||
@@ -32,14 +34,17 @@ type Local struct {
|
||||
func newLocal(c authConfig) *Local {
|
||||
// The key depends on the credentials: changing AUTH_USER or AUTH_PASS ends every session.
|
||||
m := hmac.New(sha256.New, sessionKey(c.dataDir))
|
||||
m.Write([]byte("local\x00" + c.user + "\x00" + c.pass))
|
||||
m.Write([]byte("local\x00" + c.user + "\x00" + c.pass + "\x00" + c.viewerUser + "\x00" + c.viewerPass))
|
||||
if c.loginLogo != "" {
|
||||
if _, err := os.Stat(c.loginLogo); err != nil {
|
||||
log.Printf("auth: LOGIN_LOGO: %v", err)
|
||||
}
|
||||
}
|
||||
log.Printf("local authentication enabled (user %s)", c.user)
|
||||
return &Local{user: c.user, pass: c.pass, ttl: c.sessionTTL, logo: c.loginLogo, key: m.Sum(nil)}
|
||||
if c.viewerUser != "" {
|
||||
log.Printf("local read-only account enabled (user %s)", c.viewerUser)
|
||||
}
|
||||
return &Local{user: c.user, pass: c.pass, viewerUser: c.viewerUser, viewerPass: c.viewerPass, ttl: c.sessionTTL, logo: c.loginLogo, key: m.Sum(nil)}
|
||||
}
|
||||
|
||||
func (l *Local) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -58,12 +63,17 @@ func (l *Local) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, loginPage, http.StatusFound)
|
||||
return
|
||||
}
|
||||
user, ok := l.sessionUser(r)
|
||||
s, ok := l.sessionUser(r)
|
||||
if !ok {
|
||||
if u, p, basic := r.BasicAuth(); basic && l.check(u, p) {
|
||||
user, ok = u, true
|
||||
if u, p, basic := r.BasicAuth(); basic {
|
||||
if viewer, valid := l.check(u, p); valid {
|
||||
s, ok = session{User: u, Viewer: viewer}, true
|
||||
}
|
||||
}
|
||||
}
|
||||
if ok && s.Viewer {
|
||||
r = asViewer(r)
|
||||
}
|
||||
switch {
|
||||
case r.URL.Path == loginPage:
|
||||
if ok {
|
||||
@@ -73,7 +83,7 @@ func (l *Local) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
l.next.ServeHTTP(w, r)
|
||||
case ok && r.URL.Path == "/auth/me":
|
||||
writeJSON(w, http.StatusOK, map[string]string{"mode": "local", "user": user})
|
||||
writeJSON(w, http.StatusOK, map[string]string{"mode": "local", "user": s.User, "role": roleName(s.Viewer)})
|
||||
case ok:
|
||||
l.next.ServeHTTP(w, r)
|
||||
case r.Method == http.MethodGet && !strings.HasPrefix(r.URL.Path, "/api/") && r.URL.Path != "/auth/me":
|
||||
@@ -94,7 +104,8 @@ func (l *Local) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
user, pass := r.PostFormValue("user"), r.PostFormValue("pass")
|
||||
ret := safeReturn(r.PostFormValue("r"))
|
||||
if !l.check(user, pass) {
|
||||
viewer, valid := l.check(user, pass)
|
||||
if !valid {
|
||||
log.Printf("auth: failed login for %q from %s", user, clientIP(r))
|
||||
time.Sleep(loginFailDelay)
|
||||
q := url.Values{"e": {"1"}}
|
||||
@@ -104,10 +115,10 @@ func (l *Local) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, loginPage+"?"+q.Encode(), http.StatusSeeOther)
|
||||
return
|
||||
}
|
||||
log.Printf("auth: %s logged in from %s", user, clientIP(r))
|
||||
log.Printf("auth: %s logged in from %s (%s)", user, clientIP(r), roleName(viewer))
|
||||
http.SetCookie(w, &http.Cookie{
|
||||
Name: sessionCookie,
|
||||
Value: signCookie(l.key, session{User: user, Exp: time.Now().Add(l.ttl).Unix()}),
|
||||
Value: signCookie(l.key, session{User: user, Exp: time.Now().Add(l.ttl).Unix(), Viewer: viewer}),
|
||||
Path: "/",
|
||||
MaxAge: int(l.ttl.Seconds()),
|
||||
HttpOnly: true,
|
||||
@@ -117,19 +128,30 @@ func (l *Local) handleLogin(w http.ResponseWriter, r *http.Request) {
|
||||
http.Redirect(w, r, ret, http.StatusSeeOther)
|
||||
}
|
||||
|
||||
func (l *Local) sessionUser(r *http.Request) (string, bool) {
|
||||
func (l *Local) sessionUser(r *http.Request) (session, bool) {
|
||||
var s session
|
||||
c, err := r.Cookie(sessionCookie)
|
||||
if err != nil || !verifyCookie(l.key, c.Value, &s) || time.Now().Unix() >= s.Exp {
|
||||
return "", false
|
||||
return session{}, false
|
||||
}
|
||||
return s.User, true
|
||||
return s, true
|
||||
}
|
||||
|
||||
func (l *Local) check(user, pass string) bool {
|
||||
u := subtle.ConstantTimeCompare([]byte(user), []byte(l.user))
|
||||
p := subtle.ConstantTimeCompare([]byte(pass), []byte(l.pass))
|
||||
return u&p == 1
|
||||
// check validates a user and password: the admin account, or the read-only one
|
||||
// (viewer=true) when AUTH_VIEWER_USER is set.
|
||||
func (l *Local) check(user, pass string) (viewer, ok bool) {
|
||||
if same(user, l.user) && same(pass, l.pass) {
|
||||
return false, true
|
||||
}
|
||||
if l.viewerUser != "" && same(user, l.viewerUser) && same(pass, l.viewerPass) {
|
||||
return true, true
|
||||
}
|
||||
return false, false
|
||||
}
|
||||
|
||||
// same compares in constant time, so the answer time says nothing of the secret.
|
||||
func same(a, b string) bool {
|
||||
return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
|
||||
}
|
||||
|
||||
// serveLogo sends LOGIN_LOGO; without it the login page hides the image.
|
||||
|
||||
+167
@@ -0,0 +1,167 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DBStats describes what VictoriaLogs holds (Settings > Data). Sizes are in
|
||||
// bytes; a field VictoriaLogs did not give stays at its zero value.
|
||||
type DBStats struct {
|
||||
Rows int64 `json:"rows"` // stored log lines
|
||||
DiskBytes int64 `json:"diskBytes"` // compressed data + index, on disk
|
||||
IndexBytes int64 `json:"indexBytes"` // index part of DiskBytes
|
||||
RawBytes int64 `json:"rawBytes"` // data before compression
|
||||
FreeBytes int64 `json:"freeBytes"` // free space on the VictoriaLogs volume
|
||||
Days int64 `json:"days"` // per-day partitions
|
||||
Retention string `json:"retention"` // -retentionPeriod, e.g. "30d"
|
||||
Oldest string `json:"oldest,omitempty"`
|
||||
Newest string `json:"newest,omitempty"`
|
||||
Last1h int64 `json:"last1h"`
|
||||
Last24h int64 `json:"last24h"`
|
||||
Hosts int64 `json:"hosts"`
|
||||
Apps int64 `json:"apps"`
|
||||
QueryError string `json:"queryError,omitempty"` // the LogsQL part failed, the metrics are still valid
|
||||
Updated string `json:"updated"`
|
||||
}
|
||||
|
||||
// dbStatsCache keeps the last answer for a while: the LogsQL part reads the
|
||||
// whole base, which is too heavy to repeat each time the tab is opened.
|
||||
type dbStatsCache struct {
|
||||
mu sync.Mutex
|
||||
at time.Time
|
||||
last *DBStats
|
||||
}
|
||||
|
||||
const dbStatsTTL = 30 * time.Second
|
||||
|
||||
func (c *dbStatsCache) get(ctx context.Context, s *Store, force bool) (*DBStats, error) {
|
||||
c.mu.Lock()
|
||||
defer c.mu.Unlock()
|
||||
if c.last != nil && !force && time.Since(c.at) < dbStatsTTL {
|
||||
return c.last, nil
|
||||
}
|
||||
st, err := s.DBStats(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c.last, c.at = st, time.Now()
|
||||
return st, nil
|
||||
}
|
||||
|
||||
// DBStats reads the storage metrics of VictoriaLogs (/metrics), then counts
|
||||
// with LogsQL what the metrics do not tell (period, recent volume, sources).
|
||||
func (s *Store) DBStats(ctx context.Context) (*DBStats, error) {
|
||||
st, err := s.storageMetrics(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
st.Updated = time.Now().UTC().Format(time.RFC3339)
|
||||
if st.Rows == 0 {
|
||||
return st, nil
|
||||
}
|
||||
|
||||
qctx, cancel := context.WithTimeout(ctx, 20*time.Second)
|
||||
defer cancel()
|
||||
rows, err := s.Query(qctx, "* | stats min(_time) oldest, max(_time) newest, count_uniq(host) hosts, count_uniq(app) apps")
|
||||
if err == nil && len(rows) > 0 {
|
||||
r := rows[0]
|
||||
st.Oldest, _ = r["oldest"].(string)
|
||||
st.Newest, _ = r["newest"].(string)
|
||||
st.Hosts, st.Apps = toInt(r["hosts"]), toInt(r["apps"])
|
||||
rows, err = s.Query(qctx, "_time:24h | stats count() last24h, count() if (_time:1h) last1h")
|
||||
if err == nil && len(rows) > 0 {
|
||||
st.Last24h, st.Last1h = toInt(rows[0]["last24h"]), toInt(rows[0]["last1h"])
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
st.QueryError = err.Error()
|
||||
}
|
||||
return st, nil
|
||||
}
|
||||
|
||||
func (s *Store) storageMetrics(ctx context.Context) (*DBStats, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, s.base+"/metrics", nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := s.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
|
||||
return nil, fmt.Errorf("VictoriaLogs /metrics: HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(msg)))
|
||||
}
|
||||
return parseStorageMetrics(resp.Body)
|
||||
}
|
||||
|
||||
// parseStorageMetrics picks the storage figures out of the Prometheus text
|
||||
// format of VictoriaLogs' /metrics (app/vlstorage).
|
||||
func parseStorageMetrics(r io.Reader) (*DBStats, error) {
|
||||
st := &DBStats{}
|
||||
sc := bufio.NewScanner(r)
|
||||
sc.Buffer(make([]byte, 64*1024), 1<<20)
|
||||
for sc.Scan() {
|
||||
line := sc.Text()
|
||||
if line == "" || line[0] == '#' {
|
||||
continue
|
||||
}
|
||||
name, labels, value := splitMetric(line)
|
||||
switch name {
|
||||
case "vl_storage_rows":
|
||||
st.Rows += int64(value)
|
||||
case "vl_data_size_bytes":
|
||||
st.DiskBytes += int64(value)
|
||||
if labels["type"] == "indexdb" {
|
||||
st.IndexBytes += int64(value)
|
||||
}
|
||||
case "vl_uncompressed_data_size_bytes":
|
||||
st.RawBytes += int64(value)
|
||||
case "vl_free_disk_space_bytes":
|
||||
st.FreeBytes = int64(value)
|
||||
case "vl_partitions":
|
||||
st.Days = int64(value)
|
||||
case "flag":
|
||||
if labels["name"] == "retentionPeriod" {
|
||||
st.Retention = labels["value"]
|
||||
}
|
||||
}
|
||||
}
|
||||
return st, sc.Err()
|
||||
}
|
||||
|
||||
// splitMetric splits `name{a="x",b="y"} 12` into its parts. Label values with
|
||||
// escaped quotes are not expected in the metrics read here.
|
||||
func splitMetric(line string) (string, map[string]string, float64) {
|
||||
sp := strings.LastIndexByte(line, ' ')
|
||||
if sp < 0 {
|
||||
return "", nil, 0
|
||||
}
|
||||
value, _ := strconv.ParseFloat(line[sp+1:], 64)
|
||||
head := line[:sp]
|
||||
name, rest, ok := strings.Cut(head, "{")
|
||||
if !ok {
|
||||
return head, nil, value
|
||||
}
|
||||
labels := map[string]string{}
|
||||
rest = strings.TrimSuffix(rest, "}")
|
||||
for rest != "" {
|
||||
k, v, ok := strings.Cut(rest, `="`)
|
||||
if !ok {
|
||||
break
|
||||
}
|
||||
val, after, _ := strings.Cut(v, `"`)
|
||||
labels[strings.TrimSpace(k)] = val
|
||||
rest = strings.TrimPrefix(after, ",")
|
||||
}
|
||||
return name, labels, value
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Excerpt of VictoriaLogs v1.52 /metrics.
|
||||
const vlMetrics = `# a comment
|
||||
vl_free_disk_space_bytes{path="/vlogs"} 52428800000
|
||||
vl_storage_rows{type="storage/inmemory"} 120
|
||||
vl_storage_rows{type="storage/small"} 3000
|
||||
vl_storage_rows{type="storage/big"} 1000000
|
||||
vl_partitions 12
|
||||
vl_data_size_bytes{type="indexdb"} 2048
|
||||
vl_data_size_bytes{type="storage"} 1048576
|
||||
vl_compressed_data_size_bytes{type="storage/big"} 1000000
|
||||
vl_uncompressed_data_size_bytes{type="storage/inmemory"} 4096
|
||||
vl_uncompressed_data_size_bytes{type="storage/small"} 100000
|
||||
vl_uncompressed_data_size_bytes{type="storage/big"} 20000000
|
||||
flag{name="retentionPeriod", value="30d", is_set="true"} 1
|
||||
flag{name="httpListenAddr", value=":9428", is_set="true"} 1
|
||||
`
|
||||
|
||||
func TestParseStorageMetrics(t *testing.T) {
|
||||
st, err := parseStorageMetrics(strings.NewReader(vlMetrics))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := DBStats{Rows: 1003120, DiskBytes: 1050624, IndexBytes: 2048, RawBytes: 20104096, FreeBytes: 52428800000, Days: 12, Retention: "30d"}
|
||||
if *st != want {
|
||||
t.Fatalf("got %+v\nwant %+v", *st, want)
|
||||
}
|
||||
}
|
||||
|
||||
// statsVL answers /metrics and the LogsQL queries of DBStats.
|
||||
type statsVL struct{ queries int }
|
||||
|
||||
func (v *statsVL) RoundTrip(r *http.Request) (*http.Response, error) {
|
||||
body := vlMetrics
|
||||
if r.URL.Path == "/select/logsql/query" {
|
||||
v.queries++
|
||||
_ = r.ParseForm()
|
||||
if strings.HasPrefix(r.PostForm.Get("query"), "_time:24h") {
|
||||
body = `{"last24h":"5000","last1h":"300"}` + "\n"
|
||||
} else {
|
||||
body = `{"oldest":"2026-09-21T08:00:00Z","newest":"2026-10-03T15:00:00Z","hosts":"4","apps":"17"}` + "\n"
|
||||
}
|
||||
}
|
||||
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(body)), Header: http.Header{}}, nil
|
||||
}
|
||||
|
||||
func TestDBStatsHandler(t *testing.T) {
|
||||
vl := &statsVL{}
|
||||
store := NewStore("http://vl", 10, 10, time.Second, nil)
|
||||
store.client.Transport = vl
|
||||
store.streamClient.Transport = vl
|
||||
a := &API{store: store}
|
||||
|
||||
get := func(url string) DBStats {
|
||||
rec := httptest.NewRecorder()
|
||||
a.dbStats(rec, httptest.NewRequest("GET", url, nil))
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status %d: %s", rec.Code, rec.Body)
|
||||
}
|
||||
var st DBStats
|
||||
if err := json.Unmarshal(rec.Body.Bytes(), &st); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return st
|
||||
}
|
||||
st := get("/api/dbstats")
|
||||
if st.Rows != 1003120 || st.Oldest != "2026-09-21T08:00:00Z" || st.Hosts != 4 || st.Apps != 17 || st.Last24h != 5000 || st.Last1h != 300 || st.QueryError != "" {
|
||||
t.Fatalf("unexpected stats: %+v", st)
|
||||
}
|
||||
get("/api/dbstats")
|
||||
if vl.queries != 2 {
|
||||
t.Fatalf("second call within the TTL should be cached, got %d queries", vl.queries)
|
||||
}
|
||||
get("/api/dbstats?refresh=1")
|
||||
if vl.queries != 4 {
|
||||
t.Fatalf("refresh=1 should query again, got %d queries", vl.queries)
|
||||
}
|
||||
}
|
||||
+5
-1
@@ -17,6 +17,8 @@ services:
|
||||
AUTH_MODE: ${AUTH_MODE:-local} # local (page de connexion, compte ci-dessous) ou oidc
|
||||
AUTH_USER: ${AUTH_USER:-} # vide = pas d'authentification, on delegue ca au reverse proxy traefik
|
||||
AUTH_PASS: ${AUTH_PASS:-}
|
||||
AUTH_VIEWER_USER: ${AUTH_VIEWER_USER:-} # compte en lecture seule (optionnel)
|
||||
AUTH_VIEWER_PASS: ${AUTH_VIEWER_PASS:-}
|
||||
LOGIN_LOGO: ${LOGIN_LOGO:-} # PNG affiche sur la page de connexion (chemin dans le conteneur, voir volumes)
|
||||
PRESETS_FILE: ${PRESETS_FILE:-} # prereglages de tags (defaut /data/presets.json, voir docs/presets.fr.md)
|
||||
OIDC_ISSUER: ${OIDC_ISSUER:-}
|
||||
@@ -24,10 +26,12 @@ services:
|
||||
OIDC_CLIENT_SECRET: ${OIDC_CLIENT_SECRET:-}
|
||||
OIDC_REDIRECT_URL: ${OIDC_REDIRECT_URL:-}
|
||||
OIDC_SCOPES: ${OIDC_SCOPES:-openid profile email}
|
||||
OIDC_ADMIN_GROUP: ${OIDC_ADMIN_GROUP:-} # vide = tout le monde est admin, sinon les autres sont en lecture seule
|
||||
OIDC_GROUPS_CLAIM: ${OIDC_GROUPS_CLAIM:-groups}
|
||||
SESSION_TTL: ${SESSION_TTL:-${OIDC_SESSION_TTL:-12h}} # duree de la session (local et oidc)
|
||||
RDNS: ${RDNS:-on} # resol dns
|
||||
DNS_SERVER: ${DNS_SERVER:-} # si resolv directe
|
||||
ALLOW_PURGE: ${ALLOW_PURGE:-true}
|
||||
ALLOW_PURGE: ${ALLOW_PURGE:-false} # true pour autoriser « Supprimer tous les logs »
|
||||
EXPORT_MAX: ${EXPORT_MAX:-100000}
|
||||
DOCKER_LOGS: ${DOCKER_LOGS:-on} # collecte des logs des conteneurs Docker
|
||||
DOCKER_HOST: tcp://docker-proxy:2375 # lecture seul de l'API Docker
|
||||
|
||||
@@ -61,8 +61,8 @@ type DockerManager struct {
|
||||
cfg dockerConfig
|
||||
hostName string
|
||||
containers []DockerContainer
|
||||
followers map[string]*follower // container ID -> running follower
|
||||
checkpoint map[string]time.Time // container ID -> timestamp of the last line read
|
||||
followers map[string]*follower // container ID -> running follower
|
||||
checkpoint map[string]time.Time // container ID -> timestamp of the last line stored
|
||||
dirty bool
|
||||
connected bool
|
||||
lastErr string
|
||||
@@ -389,16 +389,19 @@ func (m *DockerManager) follow(ctx context.Context, c DockerContainer) {
|
||||
}
|
||||
}
|
||||
|
||||
func sleepCtx(ctx context.Context, d time.Duration) {
|
||||
// sleepCtx waits for d, or returns false if the context ends first.
|
||||
func sleepCtx(ctx context.Context, d time.Duration) bool {
|
||||
t := time.NewTimer(d)
|
||||
defer t.Stop()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return false
|
||||
case <-t.C:
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// since returns where to resume reading: just after the last line read, or
|
||||
// since returns where to resume reading: just after the last line stored, or
|
||||
// DOCKER_BACKFILL ago for a container seen for the first time.
|
||||
func (m *DockerManager) since(id string) time.Time {
|
||||
m.mu.Lock()
|
||||
@@ -530,8 +533,6 @@ func (m *DockerManager) emit(c DockerContainer, stream string, line []byte) {
|
||||
} else if t, err := time.Parse(time.RFC3339Nano, s); err == nil {
|
||||
ts, s = t, ""
|
||||
}
|
||||
defer m.setCheckpoint(c.ID, ts)
|
||||
|
||||
s = ansiRe.ReplaceAllString(s, "")
|
||||
if !utf8.ValidString(s) {
|
||||
s = strings.ToValidUTF8(s, string(utf8.RuneError))
|
||||
@@ -575,6 +576,10 @@ func (m *DockerManager) emit(c DockerContainer, stream string, line []byte) {
|
||||
"compose_service": c.Service,
|
||||
"stream": stream,
|
||||
},
|
||||
// Docker keeps the logs: wait for room in the queue rather than drop the
|
||||
// line, and resume after it only once it is stored.
|
||||
Wait: true,
|
||||
Done: func() { m.setCheckpoint(c.ID, ts) },
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"io/fs"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"regexp"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Request guards shared by every auth mode: security headers, a cross-site
|
||||
// request check, and the read-only role.
|
||||
|
||||
type viewerKey struct{}
|
||||
|
||||
// asViewer marks the request as made by a read-only user.
|
||||
func asViewer(r *http.Request) *http.Request {
|
||||
return r.WithContext(context.WithValue(r.Context(), viewerKey{}, true))
|
||||
}
|
||||
|
||||
func isViewer(r *http.Request) bool {
|
||||
v, _ := r.Context().Value(viewerKey{}).(bool)
|
||||
return v
|
||||
}
|
||||
|
||||
func roleName(viewer bool) string {
|
||||
if viewer {
|
||||
return "viewer"
|
||||
}
|
||||
return "admin"
|
||||
}
|
||||
|
||||
func isSafeMethod(m string) bool {
|
||||
return m == http.MethodGet || m == http.MethodHead || m == http.MethodOptions
|
||||
}
|
||||
|
||||
// readOnly refuses the API calls that change something (tags, sources, purge)
|
||||
// to read-only users. Without authentication everyone is admin.
|
||||
func readOnly(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if isViewer(r) && !isSafeMethod(r.Method) && strings.HasPrefix(r.URL.Path, "/api/") {
|
||||
writeErr(w, http.StatusForbidden, &codedError{code: "read_only", msg: "read-only account: changes are reserved to administrators"})
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
// crossSite tells whether a request that changes something comes from another
|
||||
// site (a form or script on a third-party page), using the headers browsers
|
||||
// add; tools such as curl send neither and are let through.
|
||||
func crossSite(r *http.Request) bool {
|
||||
switch r.Header.Get("Sec-Fetch-Site") {
|
||||
case "same-origin", "none":
|
||||
return false
|
||||
case "":
|
||||
default: // same-site, cross-site
|
||||
return true
|
||||
}
|
||||
o := r.Header.Get("Origin")
|
||||
if o == "" {
|
||||
return false
|
||||
}
|
||||
u, err := url.Parse(o)
|
||||
return err != nil || !strings.EqualFold(u.Host, r.Host)
|
||||
}
|
||||
|
||||
// secure adds the security headers to every answer and refuses cross-site
|
||||
// changes. Without authentication it also answers /auth/me, so the UI can
|
||||
// warn that anyone on the network has full access.
|
||||
func secure(next http.Handler, csp string, authOn bool) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
h := w.Header()
|
||||
h.Set("X-Content-Type-Options", "nosniff")
|
||||
h.Set("X-Frame-Options", "DENY")
|
||||
h.Set("Referrer-Policy", "same-origin")
|
||||
h.Set("Content-Security-Policy", csp)
|
||||
if !isSafeMethod(r.Method) && crossSite(r) {
|
||||
writeErr(w, http.StatusForbidden, &codedError{code: "cross_site", msg: "cross-site request refused"})
|
||||
return
|
||||
}
|
||||
if !authOn && r.URL.Path == "/auth/me" {
|
||||
writeJSON(w, http.StatusOK, map[string]string{"mode": "none", "role": "admin"})
|
||||
return
|
||||
}
|
||||
next.ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
|
||||
var inlineScript = regexp.MustCompile(`(?s)<script>(.*?)</script>`)
|
||||
|
||||
// contentSecurityPolicy allows the UI's own files, the inline scripts of the
|
||||
// embedded pages (by hash) and the optional Bunny Fonts.
|
||||
func contentSecurityPolicy(static fs.FS) string {
|
||||
scripts := []string{"'self'"}
|
||||
for _, page := range []string{"index.html", "login.html"} {
|
||||
b, err := fs.ReadFile(static, page)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
for _, m := range inlineScript.FindAllSubmatch(b, -1) {
|
||||
sum := sha256.Sum256(m[1])
|
||||
scripts = append(scripts, "'sha256-"+base64.StdEncoding.EncodeToString(sum[:])+"'")
|
||||
}
|
||||
}
|
||||
return strings.Join([]string{
|
||||
"default-src 'self'",
|
||||
"script-src " + strings.Join(scripts, " "),
|
||||
// Inline style attributes carry the tag and project colors.
|
||||
"style-src 'self' 'unsafe-inline' https://fonts.bunny.net",
|
||||
"font-src 'self' https://fonts.bunny.net",
|
||||
"img-src 'self' data:",
|
||||
"connect-src 'self'",
|
||||
"object-src 'none'",
|
||||
"base-uri 'none'",
|
||||
"form-action 'self'",
|
||||
"frame-ancestors 'none'",
|
||||
}, "; ")
|
||||
}
|
||||
+168
@@ -0,0 +1,168 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io/fs"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/cookiejar"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var echo = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = w.Write([]byte("app " + r.URL.Path)) })
|
||||
|
||||
func TestSecureHeadersAndCrossSite(t *testing.T) {
|
||||
h := secure(echo, "default-src 'self'", false)
|
||||
cases := []struct {
|
||||
method string
|
||||
hdr map[string]string
|
||||
want int
|
||||
}{
|
||||
{"GET", map[string]string{"Sec-Fetch-Site": "cross-site"}, 200}, // reading is fine
|
||||
{"POST", nil, 200}, // curl, scripts
|
||||
{"POST", map[string]string{"Sec-Fetch-Site": "same-origin"}, 200},
|
||||
{"POST", map[string]string{"Sec-Fetch-Site": "cross-site"}, 403},
|
||||
{"DELETE", map[string]string{"Sec-Fetch-Site": "same-site"}, 403},
|
||||
{"POST", map[string]string{"Origin": "http://logs.lan:8080"}, 200},
|
||||
{"POST", map[string]string{"Origin": "https://evil.example"}, 403},
|
||||
{"PUT", map[string]string{"Origin": "null"}, 403},
|
||||
}
|
||||
for _, c := range cases {
|
||||
r := httptest.NewRequest(c.method, "http://logs.lan:8080/api/purge", nil)
|
||||
for k, v := range c.hdr {
|
||||
r.Header.Set(k, v)
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, r)
|
||||
if rec.Code != c.want {
|
||||
t.Errorf("%s %v: %d, want %d", c.method, c.hdr, rec.Code, c.want)
|
||||
}
|
||||
if rec.Header().Get("Content-Security-Policy") == "" || rec.Header().Get("X-Frame-Options") != "DENY" {
|
||||
t.Errorf("%s %v: security headers missing", c.method, c.hdr)
|
||||
}
|
||||
}
|
||||
|
||||
rec := httptest.NewRecorder()
|
||||
h.ServeHTTP(rec, httptest.NewRequest("GET", "/auth/me", nil))
|
||||
if !strings.Contains(rec.Body.String(), `"mode":"none"`) || !strings.Contains(rec.Body.String(), `"role":"admin"`) {
|
||||
t.Errorf("/auth/me without auth: %s", rec.Body)
|
||||
}
|
||||
rec = httptest.NewRecorder()
|
||||
secure(echo, "", true).ServeHTTP(rec, httptest.NewRequest("GET", "/auth/me", nil))
|
||||
if rec.Body.String() != "app /auth/me" {
|
||||
t.Errorf("/auth/me with auth answered by the guard: %s", rec.Body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContentSecurityPolicyHashesInlineScripts(t *testing.T) {
|
||||
static, _ := fs.Sub(webFS, "web")
|
||||
csp := contentSecurityPolicy(static)
|
||||
// index.html and login.html each have inline scripts.
|
||||
if n := strings.Count(csp, "'sha256-"); n < 3 {
|
||||
t.Errorf("%d script hashes in %q", n, csp)
|
||||
}
|
||||
for _, want := range []string{"frame-ancestors 'none'", "connect-src 'self'", "https://fonts.bunny.net"} {
|
||||
if !strings.Contains(csp, want) {
|
||||
t.Errorf("CSP lacks %q", want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestLocalViewerIsReadOnly(t *testing.T) {
|
||||
loginFailDelay = 0
|
||||
h, err := newAuth(authConfig{mode: "local", user: "admin", pass: "pw", viewerUser: "guest", viewerPass: "ro", dataDir: t.TempDir()}, readOnly(echo))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
jar, _ := cookiejar.New(nil)
|
||||
c := &http.Client{Jar: jar, Transport: hosts{"app.test": h}, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
|
||||
app := "http://app.test"
|
||||
|
||||
login(t, c, app, "guest", "ro", "/")
|
||||
if code, body := get(t, c, app+"/auth/me"); code != 200 || !strings.Contains(body, `"role":"viewer"`) {
|
||||
t.Fatalf("me: %d %s", code, body)
|
||||
}
|
||||
if code, _ := get(t, c, app+"/api/logs"); code != 200 {
|
||||
t.Errorf("viewer reading logs: %d", code)
|
||||
}
|
||||
res, err := c.Post(app+"/api/purge", "application/json", strings.NewReader(`{"confirm":"PURGE"}`))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.StatusCode != http.StatusForbidden {
|
||||
t.Errorf("viewer purge: %d, want 403", res.StatusCode)
|
||||
}
|
||||
|
||||
// The admin account still changes things, also through Basic auth.
|
||||
req, _ := http.NewRequest("PUT", app+"/api/syslog", strings.NewReader("{}"))
|
||||
req.SetBasicAuth("admin", "pw")
|
||||
if res, _ := (&http.Client{Transport: hosts{"app.test": h}}).Do(req); res.StatusCode != 200 {
|
||||
t.Errorf("admin change: %d", res.StatusCode)
|
||||
}
|
||||
req, _ = http.NewRequest("PUT", app+"/api/syslog", strings.NewReader("{}"))
|
||||
req.SetBasicAuth("guest", "ro")
|
||||
if res, _ := (&http.Client{Transport: hosts{"app.test": h}}).Do(req); res.StatusCode != http.StatusForbidden {
|
||||
t.Errorf("viewer change through Basic auth: %d", res.StatusCode)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOIDCAdminGroup(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
groups any
|
||||
role string
|
||||
}{
|
||||
{[]any{"staff", "/logstream-admins"}, "admin"},
|
||||
{[]any{"staff"}, "viewer"},
|
||||
{nil, "viewer"},
|
||||
} {
|
||||
idp := newFakeIdP(t)
|
||||
idp.claims = func(c map[string]any) {
|
||||
if tc.groups != nil {
|
||||
c["groups"] = tc.groups
|
||||
}
|
||||
}
|
||||
h, err := newAuth(authConfig{
|
||||
mode: "oidc", issuer: "http://idp.test/realm", clientID: "logstream", clientSecret: "s3cret",
|
||||
redirectURL: "http://app.test/auth/callback", dataDir: t.TempDir(), adminGroup: "logstream-admins",
|
||||
}, readOnly(echo))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
netw := hosts{"app.test": h, "idp.test": idp.mux}
|
||||
h.(*OIDC).client.Transport = netw
|
||||
jar, _ := cookiejar.New(nil)
|
||||
c := &http.Client{Jar: jar, Transport: netw}
|
||||
get(t, c, "http://app.test/")
|
||||
if _, body := get(t, c, "http://app.test/auth/me"); !strings.Contains(body, `"role":"`+tc.role+`"`) {
|
||||
t.Errorf("groups %v: %s, want role %s", tc.groups, body, tc.role)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestHasGroup(t *testing.T) {
|
||||
if !hasGroup("ops logstream-admins", "/logstream-admins") || !hasGroup([]any{"a", "b"}, "b") || hasGroup(42, "b") {
|
||||
t.Error("hasGroup")
|
||||
}
|
||||
}
|
||||
|
||||
func TestTCPIdleTimeout(t *testing.T) {
|
||||
a, b := net.Pipe()
|
||||
defer b.Close()
|
||||
c := idleConn{a, 30 * time.Millisecond}
|
||||
go func() { _, _ = b.Write([]byte("x")) }()
|
||||
buf := make([]byte, 1)
|
||||
if _, err := c.Read(buf); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
start := time.Now()
|
||||
if _, err := c.Read(buf); !errors.Is(err, os.ErrDeadlineExceeded) {
|
||||
t.Fatalf("silent connection: %v, want a deadline error", err)
|
||||
}
|
||||
if time.Since(start) > time.Second {
|
||||
t.Error("deadline not applied")
|
||||
}
|
||||
}
|
||||
+2
-2
@@ -153,7 +153,7 @@ func TestHistogramHandler(t *testing.T) {
|
||||
{"_time":"2000-01-01T00:00:00Z","severity":"info","hits":"9"}
|
||||
`, recent.Format(time.RFC3339), recent.Format(time.RFC3339), older.Format(time.RFC3339))
|
||||
}}
|
||||
store := NewStore("http://vl", 10, 10, time.Second)
|
||||
store := NewStore("http://vl", 10, 10, time.Second, nil)
|
||||
store.streamClient.Transport = vl
|
||||
a := &API{store: store}
|
||||
|
||||
@@ -207,7 +207,7 @@ func TestHistogramDayInParis(t *testing.T) {
|
||||
{"_time":"2026-09-21T20:00:00Z","severity":"info","hits":"100"}
|
||||
`
|
||||
}}
|
||||
store := NewStore("http://vl", 10, 10, time.Second)
|
||||
store := NewStore("http://vl", 10, 10, time.Second, nil)
|
||||
store.streamClient.Transport = vl
|
||||
a := &API{store: store}
|
||||
from := time.Date(2026, 9, 10, 0, 0, 0, 0, time.UTC)
|
||||
|
||||
@@ -327,6 +327,7 @@ func (h *HostLogs) readFile(path, name string, off int64) (int64, error) {
|
||||
}
|
||||
e.SourceType = "host"
|
||||
e.Extra = map[string]string{"log_file": "/var/log/" + name}
|
||||
e.Wait = true
|
||||
h.sink(e)
|
||||
h.count(1, 0)
|
||||
}
|
||||
@@ -400,6 +401,7 @@ func (h *HostLogs) emitJournal(je *journalEntry) {
|
||||
Proto: "journal",
|
||||
SourceType: "host",
|
||||
Extra: map[string]string{"unit": f["_SYSTEMD_UNIT"]},
|
||||
Wait: true,
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -41,6 +41,7 @@ type config struct {
|
||||
batchSize int
|
||||
queueSize int
|
||||
flushEvery time.Duration
|
||||
spoolMax int64
|
||||
}
|
||||
|
||||
func getenv(key, def string) string {
|
||||
@@ -57,6 +58,14 @@ func getenvInt(key string, def int) int {
|
||||
return def
|
||||
}
|
||||
|
||||
// getenvIntZero is getenvInt that also accepts 0 (to turn a feature off).
|
||||
func getenvIntZero(key string, def int) int {
|
||||
if v, err := strconv.Atoi(os.Getenv(key)); err == nil && v >= 0 {
|
||||
return v
|
||||
}
|
||||
return def
|
||||
}
|
||||
|
||||
func getenvBool(key string, def bool) bool {
|
||||
switch strings.ToLower(os.Getenv(key)) {
|
||||
case "1", "true", "yes", "on":
|
||||
@@ -84,6 +93,10 @@ func main() {
|
||||
mode: getenv("AUTH_MODE", "local"),
|
||||
user: os.Getenv("AUTH_USER"),
|
||||
pass: os.Getenv("AUTH_PASS"),
|
||||
viewerUser: os.Getenv("AUTH_VIEWER_USER"),
|
||||
viewerPass: os.Getenv("AUTH_VIEWER_PASS"),
|
||||
adminGroup: os.Getenv("OIDC_ADMIN_GROUP"),
|
||||
groupsClaim: os.Getenv("OIDC_GROUPS_CLAIM"),
|
||||
issuer: os.Getenv("OIDC_ISSUER"),
|
||||
clientID: os.Getenv("OIDC_CLIENT_ID"),
|
||||
clientSecret: os.Getenv("OIDC_CLIENT_SECRET"),
|
||||
@@ -95,7 +108,7 @@ func main() {
|
||||
},
|
||||
rdns: getenvBool("RDNS", true),
|
||||
dnsServer: os.Getenv("DNS_SERVER"),
|
||||
allowPurge: getenvBool("ALLOW_PURGE", true),
|
||||
allowPurge: getenvBool("ALLOW_PURGE", false),
|
||||
exportMax: getenvInt("EXPORT_MAX", 100000),
|
||||
dockerLogs: getenvBool("DOCKER_LOGS", false),
|
||||
dockerHost: getenv("DOCKER_HOST", "unix:///var/run/docker.sock"),
|
||||
@@ -105,6 +118,7 @@ func main() {
|
||||
batchSize: getenvInt("BATCH_SIZE", 1000),
|
||||
queueSize: getenvInt("QUEUE_SIZE", 100000),
|
||||
flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond,
|
||||
spoolMax: int64(getenvIntZero("SPOOL_MAX_MB", 1024)) << 20,
|
||||
}
|
||||
|
||||
cfg.auth.dataDir = cfg.dataDir
|
||||
@@ -117,7 +131,14 @@ func main() {
|
||||
log.Fatalf("tags: %v", err)
|
||||
}
|
||||
|
||||
store := NewStore(cfg.vlogsURL, cfg.batchSize, cfg.queueSize, cfg.flushEvery)
|
||||
var spool *Spool
|
||||
if cfg.spoolMax > 0 {
|
||||
if spool, err = OpenSpool(filepath.Join(cfg.dataDir, "spool"), cfg.spoolMax); err != nil {
|
||||
log.Printf("disk buffer disabled: %v", err)
|
||||
spool = nil
|
||||
}
|
||||
}
|
||||
store := NewStore(cfg.vlogsURL, cfg.batchSize, cfg.queueSize, cfg.flushEvery, spool)
|
||||
storeDone := make(chan struct{})
|
||||
go func() {
|
||||
store.Run(ctx)
|
||||
@@ -128,12 +149,19 @@ func main() {
|
||||
rdns := NewReverseDNS(cfg.rdns, cfg.dnsServer)
|
||||
sink := func(e *Entry) {
|
||||
// Host sent as an IP (or no host in the header): replace it with its DNS name.
|
||||
// A new IP waits at most 300 ms; slower lookups finish in the background.
|
||||
if name := rdns.Lookup(e.Host, 300*time.Millisecond); name != "" {
|
||||
e.HostIP, e.Host = e.Host, name
|
||||
}
|
||||
store.Enqueue(e)
|
||||
hub.Publish(e)
|
||||
// A new IP waits at most 300 ms, without holding up the listener; slower
|
||||
// lookups finish in the background.
|
||||
rdns.Resolve(e.Host, 300*time.Millisecond, func(name string) {
|
||||
if name != "" {
|
||||
e.HostIP, e.Host = e.Host, name
|
||||
}
|
||||
store.Enqueue(e)
|
||||
hub.Publish(e)
|
||||
})
|
||||
}
|
||||
tcpMaxConns = getenvInt("SYSLOG_TCP_MAX_CONNS", tcpMaxConns)
|
||||
if d := getenvDuration("SYSLOG_TCP_IDLE", tcpIdle); d > 0 {
|
||||
tcpIdle = d
|
||||
}
|
||||
// Listening errors (port already used…) are shown in Settings > Sources.
|
||||
syslogSrv := NewSyslogServer(ctx, cfg.syslogAddr, getenv("SYSLOG_PUBLIC_PORT", ""), cfg.dataDir, sink)
|
||||
@@ -161,17 +189,25 @@ func main() {
|
||||
api.Routes(mux)
|
||||
mux.Handle("GET /", http.FileServer(http.FS(static)))
|
||||
|
||||
handler, err := newAuth(cfg.auth, mux)
|
||||
handler, err := newAuth(cfg.auth, readOnly(mux))
|
||||
if err != nil {
|
||||
log.Fatalf("auth: %v", err)
|
||||
}
|
||||
if o, ok := handler.(*OIDC); ok {
|
||||
go o.checkProvider()
|
||||
}
|
||||
_, local := handler.(*Local)
|
||||
_, oidc := handler.(*OIDC)
|
||||
authOn := local || oidc
|
||||
if !authOn {
|
||||
log.Printf("warning: no authentication (AUTH_USER is empty): anyone who can reach %s can read the logs and change the settings", cfg.httpAddr)
|
||||
}
|
||||
handler = secure(handler, contentSecurityPolicy(static), authOn)
|
||||
srv := &http.Server{
|
||||
Addr: cfg.httpAddr,
|
||||
Handler: handler,
|
||||
ReadHeaderTimeout: 10 * time.Second,
|
||||
IdleTimeout: 2 * time.Minute,
|
||||
// Requests inherit the global context so SSE streams end on shutdown.
|
||||
BaseContext: func(net.Listener) context.Context { return ctx },
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"context"
|
||||
"net"
|
||||
"strings"
|
||||
@@ -15,17 +16,26 @@ type ReverseDNS struct {
|
||||
posTTL time.Duration // cache duration of a found name
|
||||
negTTL time.Duration // cache duration of "no name"
|
||||
|
||||
lookups chan struct{} // limits the lookups running at once
|
||||
waiters chan struct{} // limits the messages waiting for a lookup (Resolve)
|
||||
|
||||
mu sync.Mutex
|
||||
cache map[string]*rdnsEntry
|
||||
cache map[string]*list.Element // ip -> element of lru
|
||||
lru *list.List // *rdnsEntry, most recently used first
|
||||
}
|
||||
|
||||
type rdnsEntry struct {
|
||||
ip string
|
||||
name string
|
||||
expires time.Time
|
||||
done chan struct{} // closed once the lookup has finished
|
||||
}
|
||||
|
||||
const rdnsMaxEntries = 10000
|
||||
const (
|
||||
rdnsMaxEntries = 10000
|
||||
rdnsMaxLookups = 64
|
||||
rdnsMaxWaiters = 1024
|
||||
)
|
||||
|
||||
// NewReverseDNS uses the system resolver, or `server` ("ip" or "ip:port") when set.
|
||||
func NewReverseDNS(enabled bool, server string) *ReverseDNS {
|
||||
@@ -47,7 +57,10 @@ func NewReverseDNS(enabled bool, server string) *ReverseDNS {
|
||||
r: r,
|
||||
posTTL: time.Hour,
|
||||
negTTL: 10 * time.Minute,
|
||||
cache: make(map[string]*rdnsEntry),
|
||||
lookups: make(chan struct{}, rdnsMaxLookups),
|
||||
waiters: make(chan struct{}, rdnsMaxWaiters),
|
||||
cache: make(map[string]*list.Element),
|
||||
lru: list.New(),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,6 +73,38 @@ func isClosed(ch chan struct{}) bool {
|
||||
}
|
||||
}
|
||||
|
||||
// entry returns the cache entry of ip, starting its lookup when it is missing
|
||||
// or expired, or nil when too many lookups are already running (a flood of
|
||||
// unknown addresses). The least recently used entry makes room for a new one.
|
||||
func (d *ReverseDNS) entry(ip string) *rdnsEntry {
|
||||
d.mu.Lock()
|
||||
defer d.mu.Unlock()
|
||||
if el := d.cache[ip]; el != nil {
|
||||
e := el.Value.(*rdnsEntry)
|
||||
if !isClosed(e.done) || time.Now().Before(e.expires) {
|
||||
d.lru.MoveToFront(el)
|
||||
return e
|
||||
}
|
||||
}
|
||||
select {
|
||||
case d.lookups <- struct{}{}:
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
if el := d.cache[ip]; el != nil {
|
||||
d.lru.Remove(el)
|
||||
}
|
||||
for d.lru.Len() >= rdnsMaxEntries {
|
||||
old := d.lru.Back()
|
||||
d.lru.Remove(old)
|
||||
delete(d.cache, old.Value.(*rdnsEntry).ip)
|
||||
}
|
||||
e := &rdnsEntry{ip: ip, done: make(chan struct{})}
|
||||
d.cache[ip] = d.lru.PushFront(e)
|
||||
go d.resolve(ip, e)
|
||||
return e
|
||||
}
|
||||
|
||||
// Lookup returns the name of ip, or "" when ip is not an IP address, has no
|
||||
// PTR record, or is not resolved within `wait`. A lookup that takes longer
|
||||
// keeps running in the background and fills the cache for later calls.
|
||||
@@ -67,18 +112,10 @@ func (d *ReverseDNS) Lookup(ip string, wait time.Duration) string {
|
||||
if !d.enabled || net.ParseIP(ip) == nil {
|
||||
return ""
|
||||
}
|
||||
d.mu.Lock()
|
||||
e := d.cache[ip]
|
||||
if e == nil || (isClosed(e.done) && time.Now().After(e.expires)) {
|
||||
if len(d.cache) >= rdnsMaxEntries {
|
||||
d.cache = make(map[string]*rdnsEntry)
|
||||
}
|
||||
e = &rdnsEntry{done: make(chan struct{})}
|
||||
d.cache[ip] = e
|
||||
go d.resolve(ip, e)
|
||||
e := d.entry(ip)
|
||||
if e == nil {
|
||||
return ""
|
||||
}
|
||||
d.mu.Unlock()
|
||||
|
||||
if !isClosed(e.done) {
|
||||
timer := time.NewTimer(wait)
|
||||
defer timer.Stop()
|
||||
@@ -91,6 +128,43 @@ func (d *ReverseDNS) Lookup(ip string, wait time.Duration) string {
|
||||
return e.name
|
||||
}
|
||||
|
||||
// Resolve is Lookup without blocking the caller: fn gets the name (or "") at
|
||||
// once when it is known, otherwise from a goroutine after at most `wait`. The
|
||||
// syslog listeners use it so that a slow DNS server never delays the reading
|
||||
// of the next messages.
|
||||
func (d *ReverseDNS) Resolve(ip string, wait time.Duration, fn func(name string)) {
|
||||
if !d.enabled || net.ParseIP(ip) == nil {
|
||||
fn("")
|
||||
return
|
||||
}
|
||||
e := d.entry(ip)
|
||||
switch {
|
||||
case e == nil:
|
||||
fn("")
|
||||
return
|
||||
case isClosed(e.done):
|
||||
fn(e.name)
|
||||
return
|
||||
}
|
||||
select {
|
||||
case d.waiters <- struct{}{}:
|
||||
default:
|
||||
fn("") // too many messages waiting already
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
defer func() { <-d.waiters }()
|
||||
timer := time.NewTimer(wait)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-e.done:
|
||||
fn(e.name)
|
||||
case <-timer.C:
|
||||
fn("")
|
||||
}
|
||||
}()
|
||||
}
|
||||
|
||||
// LookupMany resolves several addresses in parallel; unresolved ones are absent.
|
||||
func (d *ReverseDNS) LookupMany(ips []string, wait time.Duration) map[string]string {
|
||||
out := make(map[string]string)
|
||||
@@ -112,6 +186,7 @@ func (d *ReverseDNS) LookupMany(ips []string, wait time.Duration) map[string]str
|
||||
}
|
||||
|
||||
func (d *ReverseDNS) resolve(ip string, e *rdnsEntry) {
|
||||
defer func() { <-d.lookups }()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
|
||||
defer cancel()
|
||||
ttl := d.negTTL
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// slowDNS never answers before the context ends.
|
||||
func slowDNS() *ReverseDNS {
|
||||
d := NewReverseDNS(true, "")
|
||||
d.r = &net.Resolver{PreferGo: true, Dial: func(ctx context.Context, _, _ string) (net.Conn, error) {
|
||||
<-ctx.Done()
|
||||
return nil, errors.New("timeout")
|
||||
}}
|
||||
return d
|
||||
}
|
||||
|
||||
func TestResolveDoesNotBlock(t *testing.T) {
|
||||
d := slowDNS()
|
||||
got := make(chan string, 1)
|
||||
start := time.Now()
|
||||
d.Resolve("192.0.2.1", 50*time.Millisecond, func(name string) { got <- name })
|
||||
if time.Since(start) > 20*time.Millisecond {
|
||||
t.Fatal("Resolve waited for the DNS server")
|
||||
}
|
||||
if name := <-got; name != "" {
|
||||
t.Errorf("name %q, want none", name)
|
||||
}
|
||||
|
||||
// Not an IP address: the callback runs at once.
|
||||
called := false
|
||||
d.Resolve("router", time.Second, func(name string) { called = name == "" })
|
||||
if !called {
|
||||
t.Error("callback not called synchronously for a host name")
|
||||
}
|
||||
}
|
||||
|
||||
func TestReverseDNSLimits(t *testing.T) {
|
||||
d := slowDNS()
|
||||
// Lookups beyond the limit are not started (and not cached).
|
||||
for i := 0; i < rdnsMaxLookups+10; i++ {
|
||||
d.Lookup(net.IPv4(10, 0, byte(i>>8), byte(i)).String(), 0)
|
||||
}
|
||||
if n := d.lru.Len(); n != rdnsMaxLookups {
|
||||
t.Errorf("%d entries, want %d", n, rdnsMaxLookups)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReverseDNSEvictsLeastRecentlyUsed(t *testing.T) {
|
||||
d := slowDNS()
|
||||
done := make(chan struct{})
|
||||
close(done)
|
||||
add := func(ip string) {
|
||||
e := &rdnsEntry{ip: ip, name: ip + ".lan", expires: time.Now().Add(time.Hour), done: done}
|
||||
d.cache[ip] = d.lru.PushFront(e)
|
||||
}
|
||||
for i := 0; i < rdnsMaxEntries; i++ {
|
||||
add(net.IPv4(10, 1, byte(i>>8), byte(i)).String())
|
||||
}
|
||||
first := net.IPv4(10, 1, 0, 0).String()
|
||||
if name := d.Lookup(first, 0); name != first+".lan" { // now the most recent
|
||||
t.Fatalf("cached name %q", name)
|
||||
}
|
||||
d.entry("192.0.2.9")
|
||||
if d.lru.Len() != rdnsMaxEntries {
|
||||
t.Errorf("%d entries, want %d", d.lru.Len(), rdnsMaxEntries)
|
||||
}
|
||||
if d.cache[first] == nil {
|
||||
t.Error("recently used entry evicted")
|
||||
}
|
||||
if d.cache[net.IPv4(10, 1, 0, 1).String()] != nil {
|
||||
t.Error("least recently used entry kept")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,136 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Spool keeps on disk the batches VictoriaLogs could not take, so that they
|
||||
// are sent later instead of being lost. Each batch is one NDJSON file named
|
||||
// <unix nanoseconds>-<lines>.ndjson; files are sent back oldest first.
|
||||
type Spool struct {
|
||||
dir string
|
||||
max int64 // maximum total size in bytes
|
||||
|
||||
mu sync.Mutex
|
||||
size int64 // bytes on disk
|
||||
lines int64 // messages on disk
|
||||
seq int64
|
||||
}
|
||||
|
||||
// errSpoolFull is returned when a batch does not fit within SPOOL_MAX_MB.
|
||||
var errSpoolFull = fmt.Errorf("disk buffer full")
|
||||
|
||||
// OpenSpool creates the directory if needed and counts the batches already
|
||||
// there (left by a previous run).
|
||||
func OpenSpool(dir string, max int64) (*Spool, error) {
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s := &Spool{dir: dir, max: max}
|
||||
files, err := s.files()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, f := range files {
|
||||
s.size += f.size
|
||||
s.lines += f.lines
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
type spoolFile struct {
|
||||
path string
|
||||
size int64
|
||||
lines int64
|
||||
}
|
||||
|
||||
// files lists the batches on disk, oldest first. Unfinished writes (.tmp)
|
||||
// are removed.
|
||||
func (s *Spool) files() ([]spoolFile, error) {
|
||||
entries, err := os.ReadDir(s.dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var out []spoolFile
|
||||
for _, e := range entries {
|
||||
name := e.Name()
|
||||
if strings.HasSuffix(name, ".tmp") {
|
||||
_ = os.Remove(filepath.Join(s.dir, name))
|
||||
continue
|
||||
}
|
||||
base, ok := strings.CutSuffix(name, ".ndjson")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
_, n, _ := strings.Cut(base, "-")
|
||||
lines, _ := strconv.ParseInt(n, 10, 64)
|
||||
info, err := e.Info()
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
out = append(out, spoolFile{path: filepath.Join(s.dir, name), size: info.Size(), lines: lines})
|
||||
}
|
||||
// The names start with a fixed-width timestamp: string order is time order.
|
||||
sort.Slice(out, func(i, j int) bool { return out[i].path < out[j].path })
|
||||
return out, nil
|
||||
}
|
||||
|
||||
// Write saves one batch of `lines` messages.
|
||||
func (s *Spool) Write(body []byte, lines int) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.size+int64(len(body)) > s.max {
|
||||
return errSpoolFull
|
||||
}
|
||||
s.seq++
|
||||
name := fmt.Sprintf("%020d%04d-%d.ndjson", time.Now().UnixNano(), s.seq%10000, lines)
|
||||
path := filepath.Join(s.dir, name)
|
||||
tmp := path + ".tmp"
|
||||
if err := os.WriteFile(tmp, body, 0o644); err != nil {
|
||||
_ = os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
if err := os.Rename(tmp, path); err != nil {
|
||||
_ = os.Remove(tmp)
|
||||
return err
|
||||
}
|
||||
s.size += int64(len(body))
|
||||
s.lines += int64(lines)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Oldest returns the oldest batch, or ok=false when the spool is empty.
|
||||
func (s *Spool) Oldest() (f spoolFile, body []byte, ok bool, err error) {
|
||||
files, err := s.files()
|
||||
if err != nil || len(files) == 0 {
|
||||
return f, nil, false, err
|
||||
}
|
||||
f = files[0]
|
||||
body, err = os.ReadFile(f.path)
|
||||
return f, body, err == nil, err
|
||||
}
|
||||
|
||||
// Remove deletes a batch once VictoriaLogs has taken it.
|
||||
func (s *Spool) Remove(f spoolFile) {
|
||||
if err := os.Remove(f.path); err != nil && !os.IsNotExist(err) {
|
||||
return
|
||||
}
|
||||
s.mu.Lock()
|
||||
s.size -= f.size
|
||||
s.lines -= f.lines
|
||||
s.mu.Unlock()
|
||||
}
|
||||
|
||||
// Pending returns the number of messages and bytes waiting on disk.
|
||||
func (s *Spool) Pending() (lines, size int64) {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
return s.lines, s.size
|
||||
}
|
||||
@@ -17,13 +17,18 @@ import (
|
||||
)
|
||||
|
||||
// Store sends messages to VictoriaLogs in batches and queries it with LogsQL.
|
||||
// Batches VictoriaLogs cannot take go to the disk spool (when enabled) and
|
||||
// are sent again, oldest first, once it answers.
|
||||
type Store struct {
|
||||
base string
|
||||
client *http.Client
|
||||
streamClient *http.Client
|
||||
in chan *Entry
|
||||
batchSize int
|
||||
flushEvery time.Duration
|
||||
in chan *Entry
|
||||
quit chan struct{} // closed on shutdown: unblocks waiting producers
|
||||
batchSize int
|
||||
flushEvery time.Duration
|
||||
spool *Spool // nil: no disk buffer
|
||||
spooled chan struct{} // wakes the replay loop up after a write to the spool
|
||||
|
||||
received atomic.Int64
|
||||
ingested atomic.Int64
|
||||
@@ -31,29 +36,42 @@ type Store struct {
|
||||
lastErr atomic.Value // string
|
||||
}
|
||||
|
||||
func NewStore(base string, batchSize, queueSize int, flushEvery time.Duration) *Store {
|
||||
func NewStore(base string, batchSize, queueSize int, flushEvery time.Duration, spool *Spool) *Store {
|
||||
s := &Store{
|
||||
base: strings.TrimRight(base, "/"),
|
||||
client: &http.Client{Timeout: 60 * time.Second},
|
||||
base: strings.TrimRight(base, "/"),
|
||||
client: &http.Client{Timeout: 60 * time.Second},
|
||||
// No global timeout: a large export can take longer than a minute. The
|
||||
// request context still cancels it when the browser goes away.
|
||||
streamClient: &http.Client{},
|
||||
in: make(chan *Entry, queueSize),
|
||||
batchSize: batchSize,
|
||||
flushEvery: flushEvery,
|
||||
in: make(chan *Entry, queueSize),
|
||||
quit: make(chan struct{}),
|
||||
batchSize: batchSize,
|
||||
flushEvery: flushEvery,
|
||||
spool: spool,
|
||||
spooled: make(chan struct{}, 1),
|
||||
}
|
||||
s.lastErr.Store("")
|
||||
return s
|
||||
}
|
||||
|
||||
// Enqueue never blocks: when the queue is full, the message is counted as dropped.
|
||||
// Enqueue adds a message to the queue. When the queue is full, a message
|
||||
// whose producer can wait (Entry.Wait: Docker, host logs) blocks until there
|
||||
// is room; any other one (syslog) is counted as dropped.
|
||||
func (s *Store) Enqueue(e *Entry) {
|
||||
s.received.Add(1)
|
||||
select {
|
||||
case s.in <- e:
|
||||
return
|
||||
default:
|
||||
s.dropped.Add(1)
|
||||
}
|
||||
if e.Wait {
|
||||
select {
|
||||
case s.in <- e:
|
||||
return
|
||||
case <-s.quit:
|
||||
}
|
||||
}
|
||||
s.dropped.Add(1)
|
||||
}
|
||||
|
||||
// Run drains the queue into VictoriaLogs until the context is cancelled,
|
||||
@@ -62,20 +80,16 @@ func (s *Store) Run(ctx context.Context) {
|
||||
ticker := time.NewTicker(s.flushEvery)
|
||||
defer ticker.Stop()
|
||||
batch := make([]*Entry, 0, s.batchSize)
|
||||
if s.spool != nil {
|
||||
go s.replay(ctx)
|
||||
}
|
||||
|
||||
flush := func() {
|
||||
if len(batch) == 0 {
|
||||
return
|
||||
flush := func(ctx context.Context) {
|
||||
if len(batch) > 0 {
|
||||
s.store(ctx, batch)
|
||||
clear(batch)
|
||||
batch = batch[:0]
|
||||
}
|
||||
if err := s.insert(batch); err != nil {
|
||||
s.dropped.Add(int64(len(batch)))
|
||||
s.lastErr.Store(err.Error())
|
||||
log.Printf("victorialogs: %d messages dropped: %v", len(batch), err)
|
||||
} else {
|
||||
s.ingested.Add(int64(len(batch)))
|
||||
s.lastErr.Store("")
|
||||
}
|
||||
batch = batch[:0]
|
||||
}
|
||||
|
||||
for {
|
||||
@@ -83,20 +97,24 @@ func (s *Store) Run(ctx context.Context) {
|
||||
case e := <-s.in:
|
||||
batch = append(batch, e)
|
||||
if len(batch) >= s.batchSize {
|
||||
flush()
|
||||
flush(ctx)
|
||||
}
|
||||
case <-ticker.C:
|
||||
flush()
|
||||
flush(ctx)
|
||||
case <-ctx.Done():
|
||||
close(s.quit)
|
||||
// The last batches get one short attempt, then go to the spool.
|
||||
end, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
for {
|
||||
select {
|
||||
case e := <-s.in:
|
||||
batch = append(batch, e)
|
||||
if len(batch) >= s.batchSize {
|
||||
flush()
|
||||
flush(end)
|
||||
}
|
||||
default:
|
||||
flush()
|
||||
flush(end)
|
||||
return
|
||||
}
|
||||
}
|
||||
@@ -104,38 +122,158 @@ func (s *Store) Run(ctx context.Context) {
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Store) insert(batch []*Entry) error {
|
||||
// store sends one batch to VictoriaLogs, or to the spool when VictoriaLogs
|
||||
// fails or older batches are still waiting there (to keep their order).
|
||||
// Entry.Done is called once the batch is stored or spooled.
|
||||
func (s *Store) store(ctx context.Context, batch []*Entry) {
|
||||
body, err := encodeBatch(batch)
|
||||
if err == nil {
|
||||
err = s.save(ctx, body, len(batch))
|
||||
}
|
||||
if err != nil {
|
||||
s.dropped.Add(int64(len(batch)))
|
||||
s.lastErr.Store(err.Error())
|
||||
log.Printf("victorialogs: %d messages dropped: %v", len(batch), err)
|
||||
return
|
||||
}
|
||||
for _, e := range batch {
|
||||
if e.Done != nil {
|
||||
e.Done()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Store) save(ctx context.Context, body []byte, lines int) error {
|
||||
if s.spool == nil {
|
||||
// No disk buffer: retry a few times, then give up.
|
||||
var err error
|
||||
for attempt := 0; attempt < 5; attempt++ {
|
||||
if attempt > 0 && !sleepCtx(ctx, time.Duration(1<<attempt)*500*time.Millisecond) { // 1s, 2s, 4s, 8s
|
||||
break
|
||||
}
|
||||
if err = s.post(ctx, body); err == nil {
|
||||
s.ingested.Add(int64(lines))
|
||||
s.lastErr.Store("")
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
var postErr error
|
||||
if waiting, _ := s.spool.Pending(); waiting == 0 {
|
||||
if postErr = s.post(ctx, body); postErr == nil {
|
||||
s.ingested.Add(int64(lines))
|
||||
s.lastErr.Store("")
|
||||
return nil
|
||||
}
|
||||
s.lastErr.Store(postErr.Error())
|
||||
}
|
||||
err := s.spool.Write(body, lines)
|
||||
if err == nil {
|
||||
select {
|
||||
case s.spooled <- struct{}{}:
|
||||
default:
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if postErr != nil {
|
||||
return fmt.Errorf("%v; %v", postErr, err)
|
||||
}
|
||||
// Spool full while older batches wait: one direct attempt before dropping.
|
||||
if postErr = s.post(ctx, body); postErr == nil {
|
||||
s.ingested.Add(int64(lines))
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("%v; %v", err, postErr)
|
||||
}
|
||||
|
||||
// replay sends the spooled batches back to VictoriaLogs, oldest first, with
|
||||
// an increasing pause (up to 30 s) while it keeps failing.
|
||||
func (s *Store) replay(ctx context.Context) {
|
||||
if n, size := s.spool.Pending(); n > 0 {
|
||||
log.Printf("spool: %d messages (%d bytes) waiting from a previous run", n, size)
|
||||
}
|
||||
backoff := time.Second
|
||||
for {
|
||||
f, body, ok, err := s.spool.Oldest()
|
||||
if err != nil {
|
||||
log.Printf("spool: %v", err)
|
||||
}
|
||||
if !ok {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case <-s.spooled:
|
||||
case <-time.After(time.Minute):
|
||||
}
|
||||
continue
|
||||
}
|
||||
err = s.post(ctx, body)
|
||||
switch {
|
||||
case err == nil:
|
||||
s.spool.Remove(f)
|
||||
s.ingested.Add(f.lines)
|
||||
s.lastErr.Store("")
|
||||
backoff = time.Second
|
||||
continue
|
||||
case isRejected(err):
|
||||
// VictoriaLogs refuses the data itself: sending it again would not help.
|
||||
s.spool.Remove(f)
|
||||
s.dropped.Add(f.lines)
|
||||
log.Printf("spool: %d messages refused by victorialogs: %v", f.lines, err)
|
||||
continue
|
||||
}
|
||||
s.lastErr.Store(err.Error())
|
||||
if !sleepCtx(ctx, backoff) {
|
||||
return
|
||||
}
|
||||
backoff = min(2*backoff, 30*time.Second)
|
||||
}
|
||||
}
|
||||
|
||||
func encodeBatch(batch []*Entry) ([]byte, error) {
|
||||
var buf bytes.Buffer
|
||||
enc := json.NewEncoder(&buf)
|
||||
enc.SetEscapeHTML(false)
|
||||
for _, e := range batch {
|
||||
if err := enc.Encode(e.Record()); err != nil {
|
||||
return err
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
u := s.base + "/insert/jsonline?_stream_fields=host,app&_msg_field=_msg&_time_field=_time"
|
||||
|
||||
var err error
|
||||
for attempt := 0; attempt < 5; attempt++ {
|
||||
if attempt > 0 {
|
||||
time.Sleep(time.Duration(1<<attempt) * 500 * time.Millisecond) // 1s, 2s, 4s, 8s
|
||||
}
|
||||
if err = s.post(u, buf.Bytes()); err == nil {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return err
|
||||
return buf.Bytes(), nil
|
||||
}
|
||||
|
||||
func (s *Store) post(u string, body []byte) error {
|
||||
resp, err := s.client.Post(u, "application/stream+json", bytes.NewReader(body))
|
||||
// insertError is an error status of VictoriaLogs to an insert.
|
||||
type insertError struct {
|
||||
status int
|
||||
msg string
|
||||
}
|
||||
|
||||
func (e *insertError) Error() string { return fmt.Sprintf("HTTP %d: %s", e.status, e.msg) }
|
||||
|
||||
// isRejected tells whether VictoriaLogs refused the data itself (4xx other
|
||||
// than 429), as opposed to being unreachable or overloaded.
|
||||
func isRejected(err error) bool {
|
||||
var ie *insertError
|
||||
return errors.As(err, &ie) && ie.status >= 400 && ie.status < 500 && ie.status != http.StatusTooManyRequests
|
||||
}
|
||||
|
||||
func (s *Store) post(ctx context.Context, body []byte) error {
|
||||
u := s.base + "/insert/jsonline?_stream_fields=host,app&_msg_field=_msg&_time_field=_time"
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/stream+json")
|
||||
resp, err := s.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode/100 != 2 {
|
||||
msg, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
|
||||
return fmt.Errorf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(msg)))
|
||||
return &insertError{status: resp.StatusCode, msg: strings.TrimSpace(string(msg))}
|
||||
}
|
||||
_, _ = io.Copy(io.Discard, resp.Body)
|
||||
return nil
|
||||
@@ -265,11 +403,15 @@ func queryStatus(err error) int {
|
||||
}
|
||||
|
||||
func (s *Store) Stats() map[string]any {
|
||||
return map[string]any{
|
||||
st := map[string]any{
|
||||
"received": s.received.Load(),
|
||||
"ingested": s.ingested.Load(),
|
||||
"dropped": s.dropped.Load(),
|
||||
"queue": len(s.in),
|
||||
"lastError": s.lastErr.Load(),
|
||||
}
|
||||
if s.spool != nil {
|
||||
st["spooled"], st["spoolBytes"] = s.spool.Pending()
|
||||
}
|
||||
return st
|
||||
}
|
||||
+216
@@ -0,0 +1,216 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// insertVL plays VictoriaLogs' insert endpoint: it answers `status` (0 = the
|
||||
// connection fails) and keeps the lines it accepted.
|
||||
type insertVL struct {
|
||||
mu sync.Mutex
|
||||
status int
|
||||
lines []string
|
||||
}
|
||||
|
||||
func (v *insertVL) set(status int) {
|
||||
v.mu.Lock()
|
||||
v.status = status
|
||||
v.mu.Unlock()
|
||||
}
|
||||
|
||||
func (v *insertVL) got() []string {
|
||||
v.mu.Lock()
|
||||
defer v.mu.Unlock()
|
||||
return append([]string(nil), v.lines...)
|
||||
}
|
||||
|
||||
func (v *insertVL) RoundTrip(r *http.Request) (*http.Response, error) {
|
||||
body, _ := io.ReadAll(r.Body)
|
||||
v.mu.Lock()
|
||||
defer v.mu.Unlock()
|
||||
if v.status == 0 {
|
||||
return nil, errors.New("connection refused")
|
||||
}
|
||||
if v.status == http.StatusOK {
|
||||
for _, l := range strings.Split(strings.TrimSpace(string(body)), "\n") {
|
||||
v.lines = append(v.lines, l)
|
||||
}
|
||||
}
|
||||
return &http.Response{StatusCode: v.status, Body: io.NopCloser(strings.NewReader("")), Header: http.Header{}}, nil
|
||||
}
|
||||
|
||||
func testEntry(msg string, done *atomic.Int64) *Entry {
|
||||
e := &Entry{Received: time.Now(), Time: time.Now(), Host: "h", App: "a", Message: msg}
|
||||
if done != nil {
|
||||
e.Done = func() { done.Add(1) }
|
||||
}
|
||||
return e
|
||||
}
|
||||
|
||||
func waitFor(t *testing.T, what string, cond func() bool) {
|
||||
t.Helper()
|
||||
for deadline := time.Now().Add(5 * time.Second); time.Now().Before(deadline); time.Sleep(10 * time.Millisecond) {
|
||||
if cond() {
|
||||
return
|
||||
}
|
||||
}
|
||||
t.Fatalf("timed out waiting for %s", what)
|
||||
}
|
||||
|
||||
func TestSpoolFiles(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
sp, err := OpenSpool(dir, 100)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := sp.Write([]byte("a\nb\n"), 2); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := sp.Write([]byte("c\n"), 1); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := sp.Write(bytes.Repeat([]byte("x"), 100), 1); !errors.Is(err, errSpoolFull) {
|
||||
t.Fatalf("write over the limit: %v, want errSpoolFull", err)
|
||||
}
|
||||
_ = os.WriteFile(filepath.Join(dir, "broken.ndjson.tmp"), []byte("z"), 0o644)
|
||||
|
||||
// A new run finds the batches left on disk and drops unfinished writes.
|
||||
sp, err = OpenSpool(dir, 100)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if n, size := sp.Pending(); n != 3 || size != 6 {
|
||||
t.Fatalf("pending %d lines %d bytes, want 3 and 6", n, size)
|
||||
}
|
||||
f, body, ok, err := sp.Oldest()
|
||||
if !ok || err != nil || string(body) != "a\nb\n" || f.lines != 2 {
|
||||
t.Fatalf("oldest: %q %+v %v %v", body, f, ok, err)
|
||||
}
|
||||
sp.Remove(f)
|
||||
if _, body, _, _ := sp.Oldest(); string(body) != "c\n" {
|
||||
t.Fatalf("next oldest %q", body)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "broken.ndjson.tmp")); !os.IsNotExist(err) {
|
||||
t.Error("unfinished write left in the spool")
|
||||
}
|
||||
}
|
||||
|
||||
func TestStoreSpoolsWhileVictoriaLogsIsDown(t *testing.T) {
|
||||
sp, err := OpenSpool(t.TempDir(), 1<<20)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
vl := &insertVL{status: 0}
|
||||
s := NewStore("http://vl", 2, 100, 20*time.Millisecond, sp)
|
||||
s.client.Transport = vl
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
go s.Run(ctx)
|
||||
|
||||
var done atomic.Int64
|
||||
for _, m := range []string{"one", "two", "three"} {
|
||||
s.Enqueue(testEntry(m, &done))
|
||||
}
|
||||
// VictoriaLogs is down: the messages are kept on disk, and acknowledged.
|
||||
waitFor(t, "3 spooled messages", func() bool { n, _ := sp.Pending(); return n == 3 })
|
||||
if done.Load() != 3 || s.dropped.Load() != 0 {
|
||||
t.Fatalf("done %d dropped %d, want 3 and 0", done.Load(), s.dropped.Load())
|
||||
}
|
||||
// While batches wait on disk, new ones queue behind them.
|
||||
vl.set(http.StatusServiceUnavailable)
|
||||
s.Enqueue(testEntry("four", &done))
|
||||
waitFor(t, "4 spooled messages", func() bool { n, _ := sp.Pending(); return n == 4 })
|
||||
|
||||
vl.set(http.StatusOK)
|
||||
waitFor(t, "the spool to drain", func() bool { n, _ := sp.Pending(); return n == 0 })
|
||||
got := strings.Join(vl.got(), "\n")
|
||||
for i, m := range []string{"one", "two", "three", "four"} {
|
||||
if !strings.Contains(got, `"_msg":"`+m+`"`) {
|
||||
t.Errorf("message %d %q not sent: %s", i, m, got)
|
||||
}
|
||||
}
|
||||
if strings.Index(got, `"one"`) > strings.Index(got, `"four"`) {
|
||||
t.Error("spooled batches sent out of order")
|
||||
}
|
||||
if s.ingested.Load() != 4 || s.lastErr.Load() != "" {
|
||||
t.Errorf("ingested %d lastErr %q", s.ingested.Load(), s.lastErr.Load())
|
||||
}
|
||||
|
||||
// Once the spool is empty, batches go straight to VictoriaLogs again.
|
||||
s.Enqueue(testEntry("five", &done))
|
||||
waitFor(t, "a direct insert", func() bool { return s.ingested.Load() == 5 })
|
||||
}
|
||||
|
||||
func TestStoreDropsRefusedSpooledBatch(t *testing.T) {
|
||||
sp, _ := OpenSpool(t.TempDir(), 1<<20)
|
||||
_ = sp.Write([]byte("{bad json\n"), 1)
|
||||
vl := &insertVL{status: http.StatusBadRequest}
|
||||
s := NewStore("http://vl", 10, 10, time.Second, sp)
|
||||
s.client.Transport = vl
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
go s.Run(ctx)
|
||||
waitFor(t, "the refused batch to be dropped", func() bool { return s.dropped.Load() == 1 })
|
||||
if n, _ := sp.Pending(); n != 0 {
|
||||
t.Errorf("%d messages left in the spool", n)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStoreWithoutSpoolDrops(t *testing.T) {
|
||||
vl := &insertVL{status: 0}
|
||||
s := NewStore("http://vl", 10, 10, time.Hour, nil)
|
||||
s.client.Transport = vl
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel() // shutdown: one attempt, no retry pauses
|
||||
var done atomic.Int64
|
||||
s.store(ctx, []*Entry{testEntry("x", &done)})
|
||||
if done.Load() != 0 || s.dropped.Load() != 1 {
|
||||
t.Errorf("done %d dropped %d, want 0 and 1", done.Load(), s.dropped.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnqueueWait(t *testing.T) {
|
||||
s := NewStore("http://vl", 10, 1, time.Hour, nil)
|
||||
s.Enqueue(testEntry("fills the queue", nil))
|
||||
s.Enqueue(testEntry("syslog: dropped", nil))
|
||||
if s.dropped.Load() != 1 {
|
||||
t.Fatalf("dropped %d, want 1", s.dropped.Load())
|
||||
}
|
||||
// A producer that can wait blocks until there is room…
|
||||
queued := make(chan struct{})
|
||||
go func() {
|
||||
e := testEntry("docker: waits", nil)
|
||||
e.Wait = true
|
||||
s.Enqueue(e)
|
||||
close(queued)
|
||||
}()
|
||||
select {
|
||||
case <-queued:
|
||||
t.Fatal("did not wait for room in the queue")
|
||||
case <-time.After(50 * time.Millisecond):
|
||||
}
|
||||
<-s.in
|
||||
<-queued
|
||||
if s.dropped.Load() != 1 {
|
||||
t.Errorf("dropped %d, want 1", s.dropped.Load())
|
||||
}
|
||||
// …or until shutdown.
|
||||
close(s.quit)
|
||||
e := testEntry("docker: shutdown", nil)
|
||||
e.Wait = true
|
||||
s.Enqueue(e)
|
||||
if s.dropped.Load() != 2 {
|
||||
t.Errorf("dropped %d after shutdown, want 2", s.dropped.Load())
|
||||
}
|
||||
}
|
||||
@@ -38,6 +38,12 @@ type Entry struct {
|
||||
|
||||
SourceType string // "syslog" or "docker"
|
||||
Extra map[string]string // additional fields (docker: container, image, …)
|
||||
|
||||
// Not stored. Wait: the producer can be slowed down when the queue is full
|
||||
// (Docker, host logs) instead of losing the message. Done: called once the
|
||||
// message is stored in VictoriaLogs or in the disk spool.
|
||||
Wait bool
|
||||
Done func()
|
||||
}
|
||||
|
||||
// Record returns the entry in the shape stored in VictoriaLogs and returned by the API.
|
||||
@@ -241,7 +247,15 @@ func serveUDP(ctx context.Context, pc net.PacketConn, sink func(*Entry)) {
|
||||
}
|
||||
}
|
||||
|
||||
// TCP limits: connections open at once, and how long a connection may stay
|
||||
// silent before it is closed (senders reconnect on their own).
|
||||
var (
|
||||
tcpMaxConns = 512
|
||||
tcpIdle = 30 * time.Minute
|
||||
)
|
||||
|
||||
func serveTCP(ctx context.Context, ln net.Listener, sink func(*Entry)) {
|
||||
slots := make(chan struct{}, tcpMaxConns)
|
||||
for {
|
||||
conn, err := ln.Accept()
|
||||
if err != nil {
|
||||
@@ -252,10 +266,32 @@ func serveTCP(ctx context.Context, ln net.Listener, sink func(*Entry)) {
|
||||
time.Sleep(100 * time.Millisecond)
|
||||
continue
|
||||
}
|
||||
go handleTCP(ctx, conn, sink)
|
||||
select {
|
||||
case slots <- struct{}{}:
|
||||
default:
|
||||
log.Printf("syslog tcp: %d connections already open, refusing %s", tcpMaxConns, conn.RemoteAddr())
|
||||
conn.Close()
|
||||
continue
|
||||
}
|
||||
go func() {
|
||||
defer func() { <-slots }()
|
||||
handleTCP(ctx, conn, sink)
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
// idleConn pushes the read deadline back before each read, so only a
|
||||
// connection that stays silent for tcpIdle is closed.
|
||||
type idleConn struct {
|
||||
net.Conn
|
||||
idle time.Duration
|
||||
}
|
||||
|
||||
func (c idleConn) Read(p []byte) (int, error) {
|
||||
_ = c.Conn.SetReadDeadline(time.Now().Add(c.idle))
|
||||
return c.Conn.Read(p)
|
||||
}
|
||||
|
||||
const maxFrame = 1 << 20
|
||||
|
||||
// handleTCP supports both RFC 6587 framings: octet counting
|
||||
@@ -266,7 +302,7 @@ func handleTCP(ctx context.Context, conn net.Conn, sink func(*Entry)) {
|
||||
defer stop()
|
||||
|
||||
src := hostOf(conn.RemoteAddr())
|
||||
r := bufio.NewReaderSize(conn, 64*1024)
|
||||
r := bufio.NewReaderSize(idleConn{conn, tcpIdle}, 64*1024)
|
||||
for {
|
||||
c, err := r.ReadByte()
|
||||
if err != nil {
|
||||
|
||||
+270
-28
@@ -33,6 +33,7 @@ const I18N = {
|
||||
skipped: (n) => `${n} messages not shown (rate too high)`,
|
||||
stats: (s) => `received ${s.received} · stored ${s.ingested} · dropped ${s.dropped} · queue ${s.queue}`,
|
||||
storageErr: 'storage: ',
|
||||
spooled: (n) => `${n} waiting on disk`,
|
||||
unreachable: 'server unreachable',
|
||||
fTime: 'timestamp', fMsg: 'message', fPid: 'pid', fSd: 'structured data',
|
||||
filterHost: 'Filter on this host', filterApp: 'Filter on this app',
|
||||
@@ -49,6 +50,9 @@ const I18N = {
|
||||
confirmDelete: (p) => `Delete tag "${p}"?`,
|
||||
confirmReset: 'Replace all tags with the defaults (warning, error)?',
|
||||
presetAria: 'Add a preset', presetPick: '+ Preset…',
|
||||
tagFilter: 'Filter tags (keyword, label or code)',
|
||||
tagCount: (n) => (n.shown === n.total ? `${n.total} tag${n.total === 1 ? '' : 's'}` : `${n.shown} / ${n.total} tags`),
|
||||
noTagMatch: 'No tag matches the filter.',
|
||||
presetAdded: (n) => (n ? `${n} tag(s) added` : 'These tags are already in the list'),
|
||||
presetsFileErr: (f) => `Invalid presets file ${f}, built-in presets used: `,
|
||||
tagsLoadErr: 'Tags: ',
|
||||
@@ -122,6 +126,16 @@ const I18N = {
|
||||
fontHelp: 'Free fonts. The built-in ones (Inconsolata Condensed, the narrowest, Iosevka and Ubuntu Mono) are served by LogStream itself and work offline; they are narrow, so more text fits on each line. The others are loaded by your browser from Bunny Fonts, a privacy-friendly European font service; without internet access, the system font is used.',
|
||||
previewMsgs: ['error: link down on eth1, carrier lost', 'warning: disk /dev/sda temperature 58°C', 'backup finished: ok (12.4 GB in 00:14:32)'],
|
||||
dangerZone: 'Danger zone',
|
||||
dbTitle: 'Database', dbRefresh: 'Refresh', dbLoading: 'Loading…',
|
||||
dbUnits: ['B', 'KB', 'MB', 'GB', 'TB'],
|
||||
dbRows: 'Stored lines', dbDisk: 'Size on disk', dbIndex: (x) => `including ${x} of index`,
|
||||
dbRaw: 'Raw size', dbRatio: (x) => `compressed ×${x}`,
|
||||
dbPeriod: 'Period', dbFromTo: (p) => `${p.from} → ${p.to}`, dbDays: (n) => `${n} day${n > 1 ? 's' : ''}`,
|
||||
dbRetention: (r) => `retention ${r}`,
|
||||
dbRecent: 'Last 24 h', dbLastHour: (n) => `${n} in the last hour`,
|
||||
dbSources: 'Sources', dbHostsApps: (p) => `${p.h} host${p.hn > 1 ? 's' : ''} · ${p.a} app${p.an > 1 ? 's' : ''}`,
|
||||
dbFree: 'Free disk space', dbEmpty: 'The database is empty.',
|
||||
dbQueryErr: 'Some figures could not be computed: ',
|
||||
purgeHelp: 'Permanently delete every stored log to start from scratch. Color tags and settings are kept.',
|
||||
purgeBtn: 'Delete all logs…',
|
||||
purgePrompt: 'This permanently deletes ALL stored logs.\n\nType PURGE to confirm:',
|
||||
@@ -131,21 +145,30 @@ const I18N = {
|
||||
purgeForbidden: 'Purging is disabled on this server (ALLOW_PURGE=false).',
|
||||
err_purge_unavailable: 'VictoriaLogs refuses deletions: start it with -delete.enable (see docker-compose.yml)',
|
||||
err_purge_forbidden: 'Purging is disabled on this server (ALLOW_PURGE=false)',
|
||||
err_read_only: 'Read-only account: changes are reserved to administrators',
|
||||
err_cross_site: 'Request refused: it comes from another site',
|
||||
authOff: 'No authentication: anyone who can reach this page can read the logs and change the settings. Set AUTH_USER / AUTH_PASS or AUTH_MODE=oidc.',
|
||||
readOnlyNote: 'Read-only account: these settings can only be changed by an administrator.',
|
||||
err_purge_confirm: 'Type PURGE to confirm',
|
||||
liveZoomed: 'Live view is not available on a zoomed range',
|
||||
connZoom: 'live paused (zoom)',
|
||||
liveZoomed: 'Live view is only available in Stream mode',
|
||||
connZoom: 'live paused (time range)',
|
||||
viewAria: 'Display mode', viewStream: 'Stream', viewPeriod: 'Time range',
|
||||
viewStreamTitle: 'Stream: the latest logs over a sliding duration, updated live',
|
||||
viewPeriodTitle: 'Time range: the logs between a start and an end date',
|
||||
perFrom: 'Start', perTo: 'End', perPrev: 'Previous time range', perNext: 'Next time range',
|
||||
perOut: 'Zoom out (twice as long)',
|
||||
histoTitle: 'Timeline',
|
||||
hScale: 'Scale', hLinear: 'Linear', hHeight: 'Height', hColor: 'Color',
|
||||
hColSeverity: 'Severity', hColIntensity: 'Intensity', hColNone: 'None',
|
||||
hRender: 'Display', hBars: 'Bars', hArea: 'Area',
|
||||
hStep: 'Division', hAuto: 'Automatic', hRefresh: 'Refresh', hOff: 'Off', hRefreshAuto: 'At each new interval',
|
||||
histoHelp: 'Click a bar to zoom on its interval, or drag across several. The √ scale keeps small volumes visible next to bursts. Intensity compares each interval with the median of the window: calm, burst (more than 3×), anomaly (more than 10×). In live mode, the last interval is updated as messages arrive.',
|
||||
histoHelp: 'Click a bar to show its interval in Time range mode, or drag across several. The √ scale keeps small volumes visible next to bursts. Intensity compares each interval with the median of the window: calm, burst (more than 3×), anomaly (more than 10×). In live mode, the last interval is updated as messages arrive.',
|
||||
hLeg_err: 'error and above', hLeg_warn: 'warning', hLeg_other: 'notice, info, debug',
|
||||
hLeg_calm: 'calm', hLeg_burst: 'burst > 3×', hLeg_anom: 'anomaly > 10×',
|
||||
hTotal: 'Total', hRatio: (x) => `${x}× the median`, hUnshown: 'not detailed (high rate)',
|
||||
hDivision: (s) => `interval ${s}`, hCapped: 'enlarged',
|
||||
hCappedTitle: 'The chosen division would exceed 300 intervals over this range: it has been enlarged.',
|
||||
hUnzoom: '× Reset zoom', unitDay: 'd',
|
||||
hUnzoom: '× Back to stream', unitDay: 'd',
|
||||
toTop: 'Back to top',
|
||||
colRcv: 'Received', colMt: 'Message time', colSev: 'Severity', colHost: 'Host', colApp: 'App', colCodes: 'Filters', colMsg: 'Message',
|
||||
codesTitle: 'Codes of the color tags found in the message',
|
||||
@@ -184,6 +207,7 @@ const I18N = {
|
||||
skipped: (n) => `${n} messages non affichés (débit trop élevé)`,
|
||||
stats: (s) => `reçus ${s.received} · stockés ${s.ingested} · perdus ${s.dropped} · file ${s.queue}`,
|
||||
storageErr: 'stockage : ',
|
||||
spooled: (n) => `${n} en attente sur disque`,
|
||||
unreachable: 'serveur injoignable',
|
||||
fTime: 'horodatage', fMsg: 'message', fPid: 'pid', fSd: 'données structurées',
|
||||
filterHost: 'Filtrer sur cet hôte', filterApp: 'Filtrer sur cette appli',
|
||||
@@ -200,6 +224,9 @@ const I18N = {
|
||||
confirmDelete: (p) => `Supprimer le tag « ${p} » ?`,
|
||||
confirmReset: 'Remplacer tous les tags par les tags par défaut (warning, error) ?',
|
||||
presetAria: 'Ajouter un préréglage', presetPick: '+ Préréglage…',
|
||||
tagFilter: 'Filtrer les tags (mot-clé, libellé ou code)',
|
||||
tagCount: (n) => (n.shown === n.total ? `${n.total} tag${n.total > 1 ? 's' : ''}` : `${n.shown} / ${n.total} tags`),
|
||||
noTagMatch: 'Aucun tag ne correspond au filtre.',
|
||||
presetAdded: (n) => (n ? `${n} tag(s) ajouté(s)` : 'Ces tags sont déjà dans la liste'),
|
||||
presetsFileErr: (f) => `Fichier de préréglages ${f} invalide, préréglages intégrés utilisés : `,
|
||||
tagsLoadErr: 'Tags : ',
|
||||
@@ -273,6 +300,16 @@ const I18N = {
|
||||
fontHelp: 'Polices libres. Les polices intégrées (Inconsolata Condensed, la plus étroite, Iosevka et Ubuntu Mono) sont servies par LogStream lui-même et fonctionnent hors ligne ; elles sont étroites, donc chaque ligne affiche plus de texte. Les autres sont chargées par votre navigateur depuis Bunny Fonts, un service européen respectueux de la vie privée ; sans accès à internet, la police du système est utilisée.',
|
||||
previewMsgs: ['error: link down on eth1, carrier lost', 'warning: disk /dev/sda temperature 58°C', 'backup finished: ok (12.4 GB in 00:14:32)'],
|
||||
dangerZone: 'Zone de danger',
|
||||
dbTitle: 'Base de données', dbRefresh: 'Actualiser', dbLoading: 'Chargement…',
|
||||
dbUnits: ['o', 'Ko', 'Mo', 'Go', 'To'],
|
||||
dbRows: 'Lignes stockées', dbDisk: 'Taille sur disque', dbIndex: (x) => `dont ${x} d'index`,
|
||||
dbRaw: 'Taille brute', dbRatio: (x) => `compression ×${x}`,
|
||||
dbPeriod: 'Période', dbFromTo: (p) => `${p.from} → ${p.to}`, dbDays: (n) => `${n} jour${n > 1 ? 's' : ''}`,
|
||||
dbRetention: (r) => `rétention ${r}`,
|
||||
dbRecent: 'Dernières 24 h', dbLastHour: (n) => `${n} sur la dernière heure`,
|
||||
dbSources: 'Sources', dbHostsApps: (p) => `${p.h} hôte${p.hn > 1 ? 's' : ''} · ${p.a} application${p.an > 1 ? 's' : ''}`,
|
||||
dbFree: 'Espace disque libre', dbEmpty: 'La base est vide.',
|
||||
dbQueryErr: 'Certains chiffres n\'ont pas pu être calculés : ',
|
||||
purgeHelp: 'Supprime définitivement tous les logs stockés pour repartir de zéro. Les tags de couleur et les réglages sont conservés.',
|
||||
purgeBtn: 'Supprimer tous les logs…',
|
||||
purgePrompt: 'Cette action supprime définitivement TOUS les logs stockés.\n\nTapez PURGE pour confirmer :',
|
||||
@@ -282,21 +319,30 @@ const I18N = {
|
||||
purgeForbidden: 'La purge est désactivée sur ce serveur (ALLOW_PURGE=false).',
|
||||
err_purge_unavailable: 'VictoriaLogs refuse les suppressions : lancez-le avec -delete.enable (voir docker-compose.yml)',
|
||||
err_purge_forbidden: 'La purge est désactivée sur ce serveur (ALLOW_PURGE=false)',
|
||||
err_read_only: 'Compte en lecture seule : les modifications sont réservées aux administrateurs',
|
||||
err_cross_site: 'Requête refusée : elle vient d\'un autre site',
|
||||
authOff: 'Aucune authentification : toute personne qui atteint cette page peut lire les logs et changer les réglages. Définissez AUTH_USER / AUTH_PASS ou AUTH_MODE=oidc.',
|
||||
readOnlyNote: 'Compte en lecture seule : seul un administrateur peut modifier ces réglages.',
|
||||
err_purge_confirm: 'Tapez PURGE pour confirmer',
|
||||
liveZoomed: 'Le direct n\'est pas disponible sur une plage zoomée',
|
||||
connZoom: 'direct en pause (zoom)',
|
||||
liveZoomed: 'Le direct n\'est disponible qu\'en mode Flux',
|
||||
connZoom: 'direct en pause (période)',
|
||||
viewAria: 'Mode d\'affichage', viewStream: 'Flux', viewPeriod: 'Période',
|
||||
viewStreamTitle: 'Flux : les derniers logs sur une durée glissante, mis à jour en direct',
|
||||
viewPeriodTitle: 'Période : les logs entre une date de début et une date de fin',
|
||||
perFrom: 'Début', perTo: 'Fin', perPrev: 'Période précédente', perNext: 'Période suivante',
|
||||
perOut: 'Élargir (deux fois plus long)',
|
||||
histoTitle: 'Frise',
|
||||
hScale: 'Échelle', hLinear: 'Linéaire', hHeight: 'Hauteur', hColor: 'Couleur',
|
||||
hColSeverity: 'Sévérité', hColIntensity: 'Intensité', hColNone: 'Aucune',
|
||||
hRender: 'Rendu', hBars: 'Barres', hArea: 'Aire',
|
||||
hStep: 'Division', hAuto: 'Automatique', hRefresh: 'Rafraîchissement', hOff: 'Désactivé', hRefreshAuto: 'À chaque nouvel intervalle',
|
||||
histoHelp: 'Cliquez sur une barre pour zoomer sur son intervalle, ou glissez sur plusieurs. L\'échelle √ garde visibles les petits volumes à côté des rafales. L\'intensité compare chaque intervalle à la médiane de la fenêtre : calme, rafale (plus de 3×), anomalie (plus de 10×). En direct, le dernier intervalle se met à jour à l\'arrivée des messages.',
|
||||
histoHelp: 'Cliquez sur une barre pour afficher son intervalle en mode Période, ou glissez sur plusieurs. L\'échelle √ garde visibles les petits volumes à côté des rafales. L\'intensité compare chaque intervalle à la médiane de la fenêtre : calme, rafale (plus de 3×), anomalie (plus de 10×). En direct, le dernier intervalle se met à jour à l\'arrivée des messages.',
|
||||
hLeg_err: 'error et plus', hLeg_warn: 'warning', hLeg_other: 'notice, info, debug',
|
||||
hLeg_calm: 'calme', hLeg_burst: 'rafale > 3×', hLeg_anom: 'anomalie > 10×',
|
||||
hTotal: 'Total', hRatio: (x) => `${x}× la médiane`, hUnshown: 'sans détail (débit élevé)',
|
||||
hDivision: (s) => `intervalle ${s}`, hCapped: 'élargi',
|
||||
hCappedTitle: 'La division choisie dépasserait 300 intervalles sur cette plage : elle a été élargie.',
|
||||
hUnzoom: '× Annuler le zoom', unitDay: 'j',
|
||||
hUnzoom: '× Revenir au flux', unitDay: 'j',
|
||||
toTop: 'Revenir en haut',
|
||||
colRcv: 'Réception', colMt: 'Heure message', colSev: 'Sévérité', colHost: 'Hôte', colApp: 'App', colCodes: 'Filtres', colMsg: 'Message',
|
||||
codesTitle: 'Codes des tags de couleur trouvés dans le message',
|
||||
@@ -457,7 +503,7 @@ const state = {
|
||||
live: store.get('live', '1') === '1',
|
||||
es: null,
|
||||
reqId: 0,
|
||||
zoom: null, // {from, to} in ms when the timeline is zoomed (range "custom")
|
||||
zoom: null, // {from, to} in ms in Time range mode (range "custom"), null in Stream mode
|
||||
tookMs: null,
|
||||
conn: ['', 'connPaused'],
|
||||
stats: null,
|
||||
@@ -486,13 +532,14 @@ function setLang(next) {
|
||||
setConn(...state.conn);
|
||||
renderList();
|
||||
updateCount(state.tookMs);
|
||||
setZoomOption();
|
||||
renderPeriod();
|
||||
renderHisto();
|
||||
renderStats();
|
||||
renderTagList();
|
||||
renderPresets();
|
||||
renderTimeSettings();
|
||||
renderPurge();
|
||||
renderDbStats();
|
||||
renderInterface();
|
||||
renderSyslog();
|
||||
renderHost();
|
||||
@@ -1398,33 +1445,115 @@ function showHistoSel() {
|
||||
sel.style.width = ((b - a + 1) / d.count) * 100 + '%';
|
||||
}
|
||||
|
||||
function setZoomOption() {
|
||||
const sel = $('#range');
|
||||
let opt = sel.querySelector('option[value="custom"]');
|
||||
if (!state.zoom) {
|
||||
if (opt) opt.remove();
|
||||
return;
|
||||
/* ---- Display mode: Stream or Time range ---- */
|
||||
|
||||
// datetime-local fields hold a wall time in the chosen time zone.
|
||||
function toWall(ms, sec) {
|
||||
const p = zoneParts(new Date(ms));
|
||||
return `${p.Y}-${p.M}-${p.D}T${p.h}:${p.m}${sec ? ':' + p.s : ''}`;
|
||||
}
|
||||
|
||||
function zoneOffsetMs(ms) {
|
||||
const m = /^([+-])(\d\d):(\d\d)$/.exec(zoneParts(new Date(ms)).off);
|
||||
return m ? (m[1] === '-' ? -1 : 1) * (m[2] * 36e5 + m[3] * 6e4) : 0;
|
||||
}
|
||||
|
||||
function fromWall(v) {
|
||||
const m = /^(\d{4})-(\d\d)-(\d\d)T(\d\d):(\d\d)(?::(\d\d))?/.exec(v || '');
|
||||
if (!m) return NaN;
|
||||
const wall = Date.UTC(m[1], m[2] - 1, m[3], m[4], m[5], m[6] || 0);
|
||||
const guess = wall - zoneOffsetMs(wall);
|
||||
return wall - zoneOffsetMs(guess); // second pass: right offset around DST changes
|
||||
}
|
||||
|
||||
const serverNow = () => Date.now() + (histo.data ? histo.data.offset : 0);
|
||||
|
||||
function renderPeriod() {
|
||||
const on = !!state.zoom;
|
||||
for (const b of $('#viewMode').querySelectorAll('[data-view]')) {
|
||||
b.setAttribute('aria-checked', String((b.dataset.view === 'period') === on));
|
||||
}
|
||||
if (!opt) { opt = new Option('', 'custom'); sel.add(opt); }
|
||||
opt.textContent = fmtSpan(state.zoom.from, state.zoom.to, state.zoom.to - state.zoom.from < 3e5);
|
||||
sel.value = 'custom';
|
||||
$('#range').hidden = on;
|
||||
$('#period').hidden = !on;
|
||||
if (!on) return;
|
||||
const { from, to } = state.zoom;
|
||||
const sec = from % 6e4 !== 0 || to % 6e4 !== 0;
|
||||
for (const [el, v] of [[$('#perFrom'), from], [$('#perTo'), to]]) {
|
||||
el.step = sec ? 1 : 60;
|
||||
el.classList.remove('bad');
|
||||
if (document.activeElement !== el) el.value = toWall(v, sec);
|
||||
}
|
||||
$('#perNext').disabled = to >= serverNow();
|
||||
}
|
||||
|
||||
function zoomTo(from, to) {
|
||||
state.zoom = { from: Math.round(from), to: Math.round(to) };
|
||||
setZoomOption();
|
||||
store.set('period', JSON.stringify(state.zoom));
|
||||
renderPeriod();
|
||||
hideHistoTip();
|
||||
refresh();
|
||||
}
|
||||
|
||||
function unzoom() {
|
||||
state.zoom = null;
|
||||
setZoomOption();
|
||||
store.set('period', '');
|
||||
renderPeriod();
|
||||
$('#range').value = store.get('range', '1h');
|
||||
if (!$('#range').value) $('#range').value = '1h';
|
||||
refresh();
|
||||
}
|
||||
|
||||
// Entering Time range mode starts from what the stream was showing.
|
||||
function enterPeriod() {
|
||||
if (state.zoom) return;
|
||||
const d = histo.data;
|
||||
if (d) return zoomTo(d.start, d.start + d.count * d.step);
|
||||
const to = Math.ceil(serverNow() / 6e4) * 6e4;
|
||||
zoomTo(to - (RANGE_MS[$('#range').value] || 36e5), to);
|
||||
}
|
||||
|
||||
$('#viewMode').addEventListener('click', (ev) => {
|
||||
const b = ev.target.closest('[data-view]');
|
||||
if (!b) return;
|
||||
if (b.dataset.view === 'period') enterPeriod();
|
||||
else if (state.zoom) unzoom();
|
||||
});
|
||||
|
||||
const onPeriodInput = debounce(() => {
|
||||
if (!state.zoom) return;
|
||||
const from = fromWall($('#perFrom').value);
|
||||
const to = fromWall($('#perTo').value);
|
||||
const ok = !isNaN(from) && !isNaN(to) && to > from;
|
||||
$('#perFrom').classList.toggle('bad', isNaN(from) || (!isNaN(to) && !ok));
|
||||
$('#perTo').classList.toggle('bad', isNaN(to) || (!isNaN(from) && !ok));
|
||||
if (ok && (from !== state.zoom.from || to !== state.zoom.to)) zoomTo(from, to);
|
||||
}, 500);
|
||||
for (const id of ['perFrom', 'perTo']) {
|
||||
$('#' + id).addEventListener('input', onPeriodInput);
|
||||
$('#' + id).addEventListener('keydown', (ev) => { if (ev.key === 'Enter') ev.target.blur(); });
|
||||
$('#' + id).addEventListener('blur', () => { if (!$('#' + id).classList.contains('bad')) renderPeriod(); });
|
||||
}
|
||||
|
||||
// ◀ ▶ move by the width of the range, − doubles it around its middle.
|
||||
function shiftPeriod(dir) {
|
||||
const { from, to } = state.zoom;
|
||||
const w = to - from;
|
||||
zoomTo(from + dir * w, to + dir * w);
|
||||
}
|
||||
$('#perPrev').addEventListener('click', () => shiftPeriod(-1));
|
||||
$('#perNext').addEventListener('click', () => shiftPeriod(1));
|
||||
$('#perOut').addEventListener('click', () => {
|
||||
const { from, to } = state.zoom;
|
||||
const w = to - from;
|
||||
let a = from - w / 2;
|
||||
let b = to + w / 2;
|
||||
const now = serverNow();
|
||||
const end = Math.max(to, now); // widen into the past rather than the future
|
||||
if (b > end) { a -= b - end; b = end; }
|
||||
const r = w >= 12e4 ? 6e4 : 1e3; // whole minutes, or seconds for short ranges
|
||||
zoomTo(Math.round(a / r) * r, Math.round(b / r) * r);
|
||||
});
|
||||
|
||||
{
|
||||
const el = $('#histo');
|
||||
el.innerHTML = '<div class="h-plot"><svg class="h-svg" preserveAspectRatio="none" aria-hidden="true"></svg>'
|
||||
@@ -1546,10 +1675,6 @@ async function refresh() {
|
||||
}
|
||||
|
||||
const onFilterChange = (ev) => {
|
||||
if (ev && ev.target.id === 'range' && state.zoom && ev.target.value !== 'custom') {
|
||||
state.zoom = null; // another range chosen: leave the zoom
|
||||
setZoomOption();
|
||||
}
|
||||
if (!state.zoom) store.set('range', $('#range').value);
|
||||
store.set('severity', $('#severity').value);
|
||||
refresh();
|
||||
@@ -1664,6 +1789,7 @@ function renderStats() {
|
||||
const n = { received: fmtNum(s.received), ingested: fmtNum(s.ingested), dropped: '%DROPPED%', queue: fmtNum(s.queue) };
|
||||
const dropped = s.dropped ? `<span class="bad">${fmtNum(s.dropped)}</span>` : fmtNum(0);
|
||||
let html = esc(t('stats', n)).replace('%DROPPED%', dropped);
|
||||
if (s.spooled) html += ` · <span class="warn">${esc(t('spooled', fmtNum(s.spooled)))}</span>`;
|
||||
if (s.lastError) html += ` · <span class="bad">${esc(t('storageErr') + s.lastError)}</span>`;
|
||||
el.innerHTML = html;
|
||||
}
|
||||
@@ -1712,7 +1838,7 @@ function onTimePrefsChange() {
|
||||
store.set('timefmt', timePrefs.fmt);
|
||||
updateTimePreview();
|
||||
renderList();
|
||||
setZoomOption();
|
||||
renderPeriod();
|
||||
refreshHisto(); // intervals are aligned on the local time
|
||||
}
|
||||
$('#tzSelect').addEventListener('change', onTimePrefsChange);
|
||||
@@ -1993,6 +2119,69 @@ setInterval(() => {
|
||||
}
|
||||
}, 4000);
|
||||
|
||||
/* ================= Database statistics ================= */
|
||||
|
||||
// 1536 -> "1,5 Ko" / "1.5 KB" (binary multiples, the usual reading for disk sizes).
|
||||
function fmtBytes(n) {
|
||||
const units = t('dbUnits');
|
||||
let i = 0;
|
||||
n = Number(n || 0);
|
||||
while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; }
|
||||
const digits = i === 0 || n >= 100 ? 0 : 1;
|
||||
return `${n.toLocaleString(t('locale'), { maximumFractionDigits: digits })} ${units[i]}`;
|
||||
}
|
||||
|
||||
const db = { stats: null, error: null, loading: false };
|
||||
|
||||
function renderDbStats() {
|
||||
const el = $('#dbStats');
|
||||
const s = db.stats;
|
||||
if (!s) {
|
||||
el.innerHTML = `<div class="db-msg${db.error ? ' bad' : ''}">${esc(db.error || t('dbLoading'))}</div>`;
|
||||
return;
|
||||
}
|
||||
const row = (k, v, sub) => `<dt>${esc(k)}</dt><dd><b>${esc(v)}</b>${sub ? ` <span class="muted">${esc(sub)}</span>` : ''}</dd>`;
|
||||
const day = (x) => fmtFull(x).replace(/[.,]\d{3}(?=\D*$)/, '');
|
||||
let html = row(t('dbRows'), fmtNum(s.rows));
|
||||
html += row(t('dbDisk'), fmtBytes(s.diskBytes), s.indexBytes ? t('dbIndex', fmtBytes(s.indexBytes)) : '');
|
||||
if (s.rawBytes) {
|
||||
const ratio = s.diskBytes ? (s.rawBytes / s.diskBytes).toLocaleString(t('locale'), { maximumFractionDigits: 1 }) : '';
|
||||
html += row(t('dbRaw'), fmtBytes(s.rawBytes), ratio ? t('dbRatio', ratio) : '');
|
||||
}
|
||||
if (s.oldest && s.newest) {
|
||||
const sub = [s.days ? t('dbDays', s.days) : '', s.retention ? t('dbRetention', s.retention) : ''].filter(Boolean).join(' · ');
|
||||
html += row(t('dbPeriod'), t('dbFromTo', { from: day(s.oldest), to: day(s.newest) }), sub);
|
||||
} else if (s.retention) {
|
||||
html += row(t('dbPeriod'), '—', t('dbRetention', s.retention));
|
||||
}
|
||||
if (s.rows) {
|
||||
html += row(t('dbRecent'), fmtNum(s.last24h), t('dbLastHour', fmtNum(s.last1h)));
|
||||
html += row(t('dbSources'), t('dbHostsApps', { h: fmtNum(s.hosts), hn: s.hosts, a: fmtNum(s.apps), an: s.apps }));
|
||||
}
|
||||
if (s.freeBytes) html += row(t('dbFree'), fmtBytes(s.freeBytes));
|
||||
if (!s.rows) html += `<div class="db-msg">${esc(t('dbEmpty'))}</div>`;
|
||||
if (s.queryError) html += `<div class="db-msg bad">${esc(t('dbQueryErr') + s.queryError)}</div>`;
|
||||
el.innerHTML = html;
|
||||
}
|
||||
|
||||
async function loadDbStats(refresh = false) {
|
||||
if (db.loading) return;
|
||||
db.loading = true;
|
||||
$('#dbRefresh').disabled = true;
|
||||
try {
|
||||
db.stats = await api('/api/dbstats' + (refresh ? '?refresh=1' : ''));
|
||||
db.error = null;
|
||||
} catch (e) {
|
||||
db.stats = null;
|
||||
db.error = e.message;
|
||||
} finally {
|
||||
db.loading = false;
|
||||
$('#dbRefresh').disabled = false;
|
||||
}
|
||||
renderDbStats();
|
||||
}
|
||||
$('#dbRefresh').addEventListener('click', () => loadDbStats(true));
|
||||
|
||||
/* ================= Purge ================= */
|
||||
|
||||
const purge = { allowed: true, running: 0, error: null, code: null, wasRunning: false, timer: null };
|
||||
@@ -2022,6 +2211,7 @@ async function loadPurgeStatus() {
|
||||
refresh();
|
||||
loadFacets();
|
||||
loadStats();
|
||||
loadDbStats(true);
|
||||
}
|
||||
purge.running = s.running || 0;
|
||||
} catch { /* shown on the next attempt */ }
|
||||
@@ -2098,8 +2288,32 @@ function tagRowHTML(tag) {
|
||||
|
||||
function renderTagList() {
|
||||
$('#tagList').innerHTML = state.tags.map(tagRowHTML).join('') || `<p class="muted small">${esc(t('noTags'))}</p>`;
|
||||
filterTagList();
|
||||
}
|
||||
|
||||
// Settings > Filters: shows only the tags whose keyword, label or code contains the filter.
|
||||
function filterTagList() {
|
||||
const q = $('#tagFilter').value.trim().toLowerCase();
|
||||
let shown = 0;
|
||||
for (const row of $('#tagList').querySelectorAll('.tag-row')) {
|
||||
const tag = state.tags.find((x) => x.id === row.dataset.id);
|
||||
const hit = !q || !tag || [tag.pattern, tag.label, tag.code].some((v) => String(v || '').toLowerCase().includes(q));
|
||||
row.hidden = !hit;
|
||||
if (hit) shown++;
|
||||
}
|
||||
let none = $('#tagList').querySelector('.tag-none');
|
||||
if (q && !shown && state.tags.length) {
|
||||
if (!none) {
|
||||
none = document.createElement('p');
|
||||
none.className = 'muted small tag-none';
|
||||
$('#tagList').append(none);
|
||||
}
|
||||
none.textContent = t('noTagMatch');
|
||||
} else if (none) none.remove();
|
||||
$('#tagCount').textContent = state.tags.length ? t('tagCount', { shown, total: state.tags.length }) : '';
|
||||
}
|
||||
$('#tagFilter').addEventListener('input', filterTagList);
|
||||
|
||||
const saveTimers = {};
|
||||
function scheduleSave(tag, rowEl) {
|
||||
clearTimeout(saveTimers[tag.id]);
|
||||
@@ -2150,6 +2364,7 @@ $('#addTag').addEventListener('click', async () => {
|
||||
try {
|
||||
const tag = await api('/api/tags', { method: 'POST', body: { pattern: t('newTag'), color, wholeWord: true, enabled: true } });
|
||||
state.tags.push(tag);
|
||||
$('#tagFilter').value = '';
|
||||
renderTagList();
|
||||
applyTags();
|
||||
const input = $('#tagList').lastElementChild.querySelector('input[type="text"]');
|
||||
@@ -2195,6 +2410,8 @@ $('#settingsBtn').addEventListener('click', () => {
|
||||
loadSyslog();
|
||||
loadHost();
|
||||
loadDocker();
|
||||
renderDbStats();
|
||||
loadDbStats();
|
||||
showSettingsTab(store.get('settingsTab', 'locale'));
|
||||
$('#settingsDlg').showModal();
|
||||
});
|
||||
@@ -2212,14 +2429,39 @@ $('#settingsDlg').addEventListener('click', (ev) => { if (ev.target === ev.curre
|
||||
applyLogFont(store.get('logFont', 'system'));
|
||||
applyLogSize(store.get('logSize', 'medium'));
|
||||
applyLogDensity(store.get('logDensity', 'normal'));
|
||||
$('#authWarnClose').addEventListener('click', () => {
|
||||
$('#authWarn').hidden = true;
|
||||
store.set('authWarnHidden', '1');
|
||||
});
|
||||
|
||||
applyLang();
|
||||
$('#range').value = store.get('range', '1h');
|
||||
if (!$('#range').value) $('#range').value = '1h';
|
||||
$('#severity').value = store.get('severity', '');
|
||||
try { // Time range mode is kept across reloads
|
||||
const z = JSON.parse(store.get('period', '') || 'null');
|
||||
if (z && z.to > z.from) state.zoom = { from: z.from, to: z.to };
|
||||
} catch { /* stream mode */ }
|
||||
renderPeriod();
|
||||
|
||||
// With a login (local or OIDC), show who is logged in and the log out button.
|
||||
// Without a login, warn that the UI is open to everyone. With a login (local or OIDC),
|
||||
// show who is logged in and the log out button, and lock the admin settings of a
|
||||
// read-only account.
|
||||
fetch('/auth/me').then((res) => (res.ok ? res.json() : null)).then((me) => {
|
||||
if (!me || !me.user) return;
|
||||
if (!me) return;
|
||||
if (me.mode === 'none' && store.get('authWarnHidden') !== '1') $('#authWarn').hidden = false;
|
||||
if (me.role === 'viewer') {
|
||||
document.body.classList.add('read-only');
|
||||
for (const p of document.querySelectorAll('[data-panel="filters"], [data-panel="sources"], [data-panel="data"]')) {
|
||||
for (const s of p.querySelectorAll('.set-section')) s.inert = true;
|
||||
const note = document.createElement('p');
|
||||
note.className = 'ro-note';
|
||||
note.dataset.i18n = 'readOnlyNote';
|
||||
note.textContent = t('readOnlyNote');
|
||||
p.prepend(note);
|
||||
}
|
||||
}
|
||||
if (!me.user) return;
|
||||
const btn = $('#logoutBtn');
|
||||
btn.hidden = false;
|
||||
btn.dataset.user = me.user;
|
||||
|
||||
+55
-15
@@ -50,6 +50,33 @@
|
||||
</header>
|
||||
|
||||
<section class="filters">
|
||||
<div id="viewMode" class="seg view-mode" role="radiogroup" data-i18n-aria="viewAria">
|
||||
<button type="button" role="radio" data-view="stream" data-i18n-title="viewStreamTitle">
|
||||
<!-- Lucide "radio" (ISC license) -->
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M4.9 19.1C1 15.2 1 8.8 4.9 4.9"/><path d="M7.8 16.2c-2.3-2.3-2.3-6.1 0-8.5"/><circle cx="12" cy="12" r="2"/><path d="M16.2 7.8c2.3 2.3 2.3 6.1 0 8.5"/><path d="M19.1 4.9C23 8.8 23 15.1 19.1 19"/></svg>
|
||||
<span data-i18n="viewStream">Stream</span>
|
||||
</button>
|
||||
<button type="button" role="radio" data-view="period" data-i18n-title="viewPeriodTitle">
|
||||
<!-- Lucide "calendar-range" (ISC license) -->
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true"><rect width="18" height="18" x="3" y="4" rx="2"/><path d="M16 2v4M3 10h18M8 2v4M17 14h-6M13 18H7M7 14h.01M17 18h.01"/></svg>
|
||||
<span data-i18n="viewPeriod">Time range</span>
|
||||
</button>
|
||||
</div>
|
||||
<div id="period" class="period" hidden>
|
||||
<button id="perPrev" class="btn tool" type="button" data-i18n-title="perPrev" data-i18n-aria="perPrev">
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="m15 18-6-6 6-6"/></svg>
|
||||
</button>
|
||||
<input id="perFrom" type="datetime-local" data-i18n-title="perFrom" data-i18n-aria="perFrom">
|
||||
<span class="muted" aria-hidden="true">→</span>
|
||||
<input id="perTo" type="datetime-local" data-i18n-title="perTo" data-i18n-aria="perTo">
|
||||
<button id="perNext" class="btn tool" type="button" data-i18n-title="perNext" data-i18n-aria="perNext">
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="m9 18 6-6-6-6"/></svg>
|
||||
</button>
|
||||
<button id="perOut" class="btn tool" type="button" data-i18n-title="perOut" data-i18n-aria="perOut">
|
||||
<!-- Lucide "zoom-out" (ISC license) -->
|
||||
<svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="8"/><path d="m21 21-4.3-4.3M8 11h6"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
<select id="range" data-i18n-aria="rangeAria">
|
||||
<option value="5m" data-i18n="r5m">5 min</option>
|
||||
<option value="15m" data-i18n="r15m">15 min</option>
|
||||
@@ -97,6 +124,12 @@
|
||||
|
||||
<section id="histo" class="histo" data-i18n-aria="histoAria" hidden></section>
|
||||
|
||||
<div id="authWarn" class="auth-warn" role="status" hidden>
|
||||
<span data-i18n="authOff"></span>
|
||||
<button id="authWarnClose" class="icon-btn" type="button" data-i18n-aria="close">
|
||||
<svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
<div id="error" class="error-banner" hidden></div>
|
||||
<button id="newPill" class="pill" type="button" hidden></button>
|
||||
|
||||
@@ -183,20 +216,22 @@
|
||||
</div>
|
||||
|
||||
<!-- Filters -->
|
||||
<div class="set-panel" role="tabpanel" data-panel="filters" hidden>
|
||||
<section class="set-section">
|
||||
<h3 data-i18n="tagsTitle">Color tags</h3>
|
||||
<div class="set-panel wide" role="tabpanel" data-panel="filters" hidden>
|
||||
<section class="set-section tags-section">
|
||||
<div class="sec-head">
|
||||
<h3 data-i18n="tagsTitle">Color tags</h3>
|
||||
<span id="tagCount" class="muted small"></span>
|
||||
</div>
|
||||
<p class="muted small" data-i18n="tagsHelp"></p>
|
||||
<div id="tagList" class="tag-list"></div>
|
||||
<footer>
|
||||
<span class="tag-add">
|
||||
<button id="addTag" class="btn primary" type="button" data-i18n="addTag">+ Add tag</button>
|
||||
<select id="presetTags" class="field" aria-label="Presets" data-i18n-aria="presetAria">
|
||||
<option value="" data-i18n="presetPick">+ Preset…</option>
|
||||
</select>
|
||||
</span>
|
||||
<div class="tag-toolbar">
|
||||
<button id="addTag" class="btn primary" type="button" data-i18n="addTag">+ Add tag</button>
|
||||
<select id="presetTags" class="field" aria-label="Presets" data-i18n-aria="presetAria">
|
||||
<option value="" data-i18n="presetPick">+ Preset…</option>
|
||||
</select>
|
||||
<input id="tagFilter" type="search" class="dk-filter" autocomplete="off" spellcheck="false" data-i18n-ph="tagFilter" data-i18n-aria="tagFilter">
|
||||
<button id="resetTags" class="btn ghost" type="button" data-i18n="resetTags">Restore default tags</button>
|
||||
</footer>
|
||||
</div>
|
||||
<div id="tagList" class="tag-list"></div>
|
||||
</section>
|
||||
</div>
|
||||
|
||||
@@ -230,7 +265,7 @@
|
||||
<p class="muted small" data-i18n="hostHelp"></p>
|
||||
</section>
|
||||
|
||||
<section class="set-section">
|
||||
<section class="set-section span">
|
||||
<h3 data-i18n="dockerTitle">Docker containers</h3>
|
||||
<p id="dockerStatus" class="docker-status"></p>
|
||||
<div id="dockerBody" hidden>
|
||||
@@ -267,7 +302,7 @@
|
||||
</div>
|
||||
<p class="muted small hint" data-i18n="themeHelp"></p>
|
||||
</section>
|
||||
<section class="set-section">
|
||||
<section class="set-section span">
|
||||
<h3 data-i18n="logDisplay">Log display</h3>
|
||||
<div class="field-grid">
|
||||
<span class="lbl" data-i18n="fontSize">Font size</span>
|
||||
@@ -328,9 +363,14 @@
|
||||
</div>
|
||||
|
||||
<!-- Data -->
|
||||
<div class="set-panel" role="tabpanel" data-panel="data" hidden>
|
||||
<div class="set-panel wide" role="tabpanel" data-panel="data" hidden>
|
||||
<section class="set-section danger">
|
||||
<h3 data-i18n="dangerZone">Danger zone</h3>
|
||||
<div class="db-head">
|
||||
<span class="lbl" data-i18n="dbTitle">Database</span>
|
||||
<button id="dbRefresh" class="link-btn" type="button" data-i18n="dbRefresh">Refresh</button>
|
||||
</div>
|
||||
<dl id="dbStats" class="db-stats"></dl>
|
||||
<p class="muted small" data-i18n="purgeHelp"></p>
|
||||
<div class="purge-row">
|
||||
<button id="purgeBtn" class="btn danger" type="button" data-i18n="purgeBtn">Delete all logs…</button>
|
||||
|
||||
+104
-27
@@ -240,6 +240,19 @@ body.busy .progress::after {
|
||||
font-size: 13px; max-width: 220px;
|
||||
}
|
||||
.filters select.set { border-color: var(--accent); background-color: var(--accent-soft); }
|
||||
/* Display mode: Stream or Time range, then the start → end fields of the range */
|
||||
.view-mode { padding: 2px; gap: 2px; border-radius: 9px; }
|
||||
.view-mode button { display: inline-flex; align-items: center; gap: 6px; height: 26px; padding: 0 10px; }
|
||||
.view-mode svg { width: 15px; height: 15px; }
|
||||
.period { display: inline-flex; align-items: center; gap: 6px; }
|
||||
.period .btn.tool { width: 32px; padding: 0; justify-content: center; }
|
||||
.period input {
|
||||
height: 32px; padding: 0 8px;
|
||||
border: 1px solid var(--accent); border-radius: 8px; background: var(--accent-soft);
|
||||
font-size: 13px; font-variant-numeric: tabular-nums; color-scheme: light dark;
|
||||
}
|
||||
.period input:focus { outline: 0; box-shadow: 0 0 0 3px var(--accent-soft); }
|
||||
.period input.bad { border-color: var(--sev-err); background: color-mix(in srgb, var(--sev-err) 10%, transparent); }
|
||||
.spacer { flex: 1; }
|
||||
#count { font-size: 12.5px; font-variant-numeric: tabular-nums; }
|
||||
|
||||
@@ -433,6 +446,22 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
|
||||
font-family: var(--mono); font-size: 12.5px; white-space: pre-wrap;
|
||||
}
|
||||
|
||||
.auth-warn {
|
||||
display: flex; align-items: center; gap: 10px;
|
||||
margin: 6px 20px 0; padding: 6px 8px 6px 14px;
|
||||
border: 1px solid color-mix(in srgb, var(--sev-warning) 45%, transparent);
|
||||
background: color-mix(in srgb, var(--sev-warning) 12%, transparent);
|
||||
color: var(--text); border-radius: var(--radius); font-size: 12.5px;
|
||||
}
|
||||
.auth-warn[hidden] { display: none; }
|
||||
.auth-warn span { flex: 1; }
|
||||
.auth-warn .icon-btn { width: 26px; height: 26px; flex: none; }
|
||||
.ro-note {
|
||||
margin: 0 0 12px; padding: 8px 12px; border-radius: var(--radius); font-size: 12.5px;
|
||||
background: color-mix(in srgb, var(--accent) 10%, transparent); color: var(--text);
|
||||
}
|
||||
.read-only .set-panel .set-section[inert] { opacity: .55; }
|
||||
|
||||
.pill {
|
||||
position: fixed; top: 70px; left: 50%; transform: translateX(-50%); z-index: 30;
|
||||
border: 0; border-radius: 999px; padding: 7px 16px;
|
||||
@@ -452,6 +481,7 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
|
||||
.conn.ok::before { color: var(--sev-info); }
|
||||
.conn.ko::before { color: var(--sev-err); }
|
||||
#stats .bad { color: var(--sev-err); }
|
||||
#stats .warn { color: var(--sev-warning); }
|
||||
/* "Back to top", at the right end of the status bar: never over a log row */
|
||||
.to-top { width: 24px; height: 24px; margin: -5px -6px -5px auto; flex: none; } /* no taller bar */
|
||||
.to-top svg { width: 16px; height: 16px; }
|
||||
@@ -465,12 +495,12 @@ mark.hit { background: var(--hit); color: inherit; border-radius: 3px; padding:
|
||||
|
||||
/* ---------- Settings dialog ---------- */
|
||||
dialog.settings {
|
||||
width: min(900px, calc(100vw - 32px));
|
||||
height: min(660px, calc(100vh - 64px)); max-height: none;
|
||||
width: min(1280px, calc(100vw - 32px));
|
||||
height: calc(100vh - 48px); max-height: 960px;
|
||||
}
|
||||
dialog.settings .dlg { display: flex; flex-direction: column; height: 100%; padding: 0; }
|
||||
dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px solid var(--border); }
|
||||
.set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: 200px minmax(0, 1fr); }
|
||||
.set-body { flex: 1; min-height: 0; display: grid; grid-template-columns: 176px minmax(0, 1fr); }
|
||||
.set-nav {
|
||||
display: flex; flex-direction: column; gap: 2px;
|
||||
padding: 12px 10px; border-right: 1px solid var(--border);
|
||||
@@ -485,8 +515,19 @@ dialog.settings .dlg > header { padding: 14px 16px 14px 22px; border-bottom: 1px
|
||||
.set-nav button:hover { background: var(--panel-2); color: var(--text); }
|
||||
.set-nav button[aria-selected="true"] { background: var(--accent-soft); color: var(--accent); }
|
||||
.set-nav svg { width: 18px; height: 18px; flex: none; }
|
||||
.set-panels { overflow-y: auto; padding: 2px 24px 22px; }
|
||||
.set-panel > .set-section:first-child { margin-top: 16px; }
|
||||
.set-panels { overflow-y: auto; padding: 16px 20px 20px; }
|
||||
/* Sections are cards laid out in columns when there is room; .span ones take the full width. */
|
||||
.set-panel { display: grid; grid-template-columns: repeat(auto-fit, minmax(380px, 1fr)); gap: 14px; align-items: start; grid-auto-flow: row dense; }
|
||||
.set-panel[hidden] { display: none; }
|
||||
.set-panel.wide { grid-template-columns: minmax(0, 1fr); }
|
||||
.set-panel > .set-section, .set-panel > .set-section + .set-section {
|
||||
margin: 0; padding: 14px 16px; min-width: 0;
|
||||
border: 1px solid var(--border); border-radius: 12px;
|
||||
background: color-mix(in srgb, var(--panel-2) 35%, var(--panel));
|
||||
}
|
||||
.set-panel > .set-section.span { grid-column: 1 / -1; }
|
||||
.set-panel > .ro-note { grid-column: 1 / -1; margin: 0; }
|
||||
.sec-head { display: flex; align-items: baseline; justify-content: space-between; gap: 12px; }
|
||||
.hint { margin-top: 8px !important; }
|
||||
.lbl { font-size: 13px; font-weight: 550; }
|
||||
.set-panel .preview-list { margin: 14px 0 10px; pointer-events: none; }
|
||||
@@ -531,6 +572,17 @@ select.field:focus { outline: 0; border-color: var(--accent); box-shadow: 0 0 0
|
||||
background: var(--panel-2); font-family: var(--mono); font-size: 12.5px;
|
||||
}
|
||||
.set-section.danger h3 { color: var(--sev-err); }
|
||||
.db-head { display: flex; align-items: baseline; justify-content: space-between; gap: 10px; margin: 4px 0 6px; }
|
||||
.db-stats {
|
||||
display: grid; grid-template-columns: max-content minmax(0, 1fr); gap: 5px 14px;
|
||||
margin: 0 0 14px; padding: 10px 12px; border-radius: 8px; background: var(--panel-2); font-size: 12.5px;
|
||||
}
|
||||
.db-stats dt { color: var(--muted); }
|
||||
.db-stats dd { margin: 0; min-width: 0; overflow-wrap: anywhere; }
|
||||
.db-stats dd b { font-weight: 600; font-variant-numeric: tabular-nums; }
|
||||
.db-msg { grid-column: 1 / -1; color: var(--muted); }
|
||||
.db-msg.bad { color: var(--sev-err); }
|
||||
.link-btn:disabled { opacity: .5; cursor: default; text-decoration: none; }
|
||||
.purge-row { display: flex; flex-wrap: wrap; align-items: center; gap: 10px; margin-top: 10px; }
|
||||
.btn.danger { color: var(--sev-err); border-color: color-mix(in srgb, var(--sev-err) 45%, var(--border)); }
|
||||
.btn.danger:hover:not(:disabled) { background: var(--sev-err); border-color: var(--sev-err); color: #fff; }
|
||||
@@ -605,9 +657,19 @@ input.switch:checked::after { transform: translateX(14px); }
|
||||
input.switch:disabled { cursor: not-allowed; }
|
||||
input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
|
||||
|
||||
.set-section footer { display: flex; flex-wrap: wrap; gap: 8px; justify-content: space-between; margin-top: 14px; }
|
||||
.tag-add { display: flex; flex-wrap: wrap; gap: 8px; }
|
||||
.tag-add select.field { width: auto; }
|
||||
/* Settings > Filters: toolbar kept in view while the list scrolls, dense rows. */
|
||||
.tags-section { padding-top: 12px !important; }
|
||||
.tag-toolbar {
|
||||
position: sticky; top: -16px; z-index: 2;
|
||||
display: flex; flex-wrap: wrap; align-items: center; gap: 8px;
|
||||
margin: 10px -16px 0; padding: 8px 16px;
|
||||
background: color-mix(in srgb, var(--panel-2) 35%, var(--panel));
|
||||
border-bottom: 1px solid var(--border);
|
||||
}
|
||||
.tag-toolbar .btn { height: 30px; }
|
||||
.tag-toolbar select.field { width: auto; height: 30px; }
|
||||
.tag-toolbar .dk-filter { flex: 1 1 220px; height: 30px; }
|
||||
.tag-toolbar #resetTags { margin-left: auto; }
|
||||
|
||||
.seg { display: inline-flex; padding: 3px; gap: 3px; background: var(--panel-2); border: 1px solid var(--border); border-radius: 10px; }
|
||||
.seg button {
|
||||
@@ -617,34 +679,38 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
|
||||
.seg button:hover { color: var(--text); }
|
||||
.seg button[aria-checked="true"] { background: var(--panel); color: var(--accent); box-shadow: 0 1px 3px rgba(0, 0, 0, .12); }
|
||||
|
||||
.tag-list { display: flex; flex-direction: column; gap: 8px; margin-top: 12px; }
|
||||
/* One wide column: long regular expressions stay readable. */
|
||||
.tag-list { display: flex; flex-direction: column; gap: 3px; margin-top: 10px; }
|
||||
.tag-row {
|
||||
display: grid; align-items: center; gap: 8px 10px;
|
||||
grid-template-columns: auto 38px minmax(8rem, 1fr) 7.5rem auto 36px;
|
||||
padding: 8px 10px; border: 1px solid var(--border); border-radius: 10px;
|
||||
background: var(--panel-2);
|
||||
display: grid; align-items: center; gap: 4px 6px;
|
||||
grid-template-columns: 2.4em 30px minmax(7rem, 1fr) 10rem auto 26px;
|
||||
padding: 3px 4px 3px 6px; border: 1px solid var(--border); border-radius: 8px;
|
||||
background: var(--panel);
|
||||
}
|
||||
.tag-row[hidden] { display: none; }
|
||||
.tag-row.off { opacity: .55; }
|
||||
.tag-row .tag-code { font-size: 12px; padding: 3px 6px; cursor: default; }
|
||||
.tag-row .tag-code { font-size: 11px; padding: 2px 4px; cursor: default; text-align: center; }
|
||||
.tag-row input[type="color"] {
|
||||
width: 38px; height: 32px; padding: 0; border: 1px solid var(--border); border-radius: 8px;
|
||||
width: 30px; height: 26px; padding: 0; border: 1px solid var(--border); border-radius: 6px;
|
||||
background: none; cursor: pointer;
|
||||
}
|
||||
.tag-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 3px; }
|
||||
.tag-row input[type="color"]::-webkit-color-swatch-wrapper { padding: 2px; }
|
||||
.tag-row input[type="color"]::-webkit-color-swatch { border: 0; border-radius: 5px; }
|
||||
.tag-row input[type="text"] {
|
||||
height: 32px; padding: 0 10px; min-width: 0;
|
||||
border: 1px solid var(--border); border-radius: 8px; background: var(--panel);
|
||||
font-family: var(--mono); font-size: 13px; outline: 0;
|
||||
height: 26px; padding: 0 8px; min-width: 0;
|
||||
border: 1px solid var(--border); border-radius: 6px; background: var(--panel);
|
||||
font-family: var(--mono); font-size: 12.5px; outline: 0;
|
||||
}
|
||||
.tag-row input[type="text"]:focus { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
|
||||
.tag-row .preview { font-family: var(--mono); font-size: 12.5px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; text-align: center; }
|
||||
.tag-row .opts { display: flex; gap: 4px; }
|
||||
.tag-row .preview { font-family: var(--mono); font-size: 12px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; text-align: center; }
|
||||
.tag-row .opts { display: flex; gap: 3px; }
|
||||
.opt {
|
||||
display: inline-flex; align-items: center; height: 28px; padding: 0 8px;
|
||||
border: 1px solid var(--border); border-radius: 7px; background: var(--panel);
|
||||
font-size: 11.5px; font-weight: 600; color: var(--muted); cursor: pointer; user-select: none;
|
||||
display: inline-flex; align-items: center; height: 24px; padding: 0 6px;
|
||||
border: 1px solid var(--border); border-radius: 6px; background: var(--panel);
|
||||
font-size: 11px; font-weight: 600; color: var(--muted); cursor: pointer; user-select: none;
|
||||
}
|
||||
.tag-row .icon-btn { width: 26px; height: 26px; border-radius: 6px; }
|
||||
.tag-row .icon-btn svg { width: 15px; height: 15px; }
|
||||
.opt input { display: none; }
|
||||
.opt:has(input:checked) { color: var(--accent); border-color: var(--accent); background: var(--accent-soft); }
|
||||
.tag-row .err { grid-column: 1 / -1; color: var(--sev-err); font-size: 12px; }
|
||||
@@ -658,11 +724,18 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
|
||||
.live .lbl { display: none; }
|
||||
.filters { padding: 10px 16px 6px; }
|
||||
.filters select { flex: 1 1 calc(50% - 8px); max-width: none; min-width: 0; }
|
||||
.view-mode { flex: 1 1 100%; }
|
||||
.view-mode button { flex: 1; justify-content: center; }
|
||||
/* Phones: the two dates on one line, the ◀ ▶ − buttons below */
|
||||
.period { flex: 1 1 100%; flex-wrap: wrap; }
|
||||
.period > span { display: none; }
|
||||
.period input { flex: 1 1 calc(50% - 3px); min-width: 0; }
|
||||
.period .btn.tool { order: 1; flex: 1; }
|
||||
.spacer { display: none; }
|
||||
#count { flex: 1 1 50%; min-width: 0; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||
.histo { padding: 0 16px 6px; }
|
||||
.h-leg { display: none; }
|
||||
.list, .error-banner, .table { margin-left: 16px; margin-right: 16px; }
|
||||
.list, .error-banner, .auth-warn, .table { margin-left: 16px; margin-right: 16px; }
|
||||
/* Phones: no columns, two lines per log (layout below); the widths do not apply */
|
||||
.table { display: block; }
|
||||
.table .list { display: block; margin: 0; }
|
||||
@@ -694,13 +767,17 @@ input.switch:focus-visible { outline: 2px solid var(--accent); outline-offset: 2
|
||||
font-size: 11.5px; text-align: center;
|
||||
}
|
||||
.set-nav button span { max-width: 100%; overflow: hidden; text-overflow: ellipsis; }
|
||||
.set-panels { padding: 0 16px 18px; }
|
||||
.set-panels { padding: 12px 12px 18px; }
|
||||
.set-panel { grid-template-columns: minmax(0, 1fr); gap: 10px; }
|
||||
.set-panel > .set-section, .set-panel > .set-section + .set-section { padding: 12px; }
|
||||
.tag-toolbar { position: static; margin: 10px -12px 0; padding: 8px 12px; }
|
||||
.tag-toolbar #resetTags { margin-left: 0; }
|
||||
#sizeSwitch, #densitySwitch { display: flex; }
|
||||
#sizeSwitch button, #densitySwitch button { flex: 1; padding: 0 4px; white-space: nowrap; font-size: 12px; }
|
||||
:root[data-density="compact"] .row { padding-top: 3px; padding-bottom: 3px; }
|
||||
.field-grid select { margin-bottom: 6px; }
|
||||
.status { padding: 6px 16px; }
|
||||
.tag-row { grid-template-columns: auto 38px 1fr 36px; }
|
||||
.tag-row { grid-template-columns: auto 30px 1fr 26px; }
|
||||
.tag-row .preview { display: none; }
|
||||
.tag-row .opts { grid-column: 1 / -1; grid-row: 2; flex-wrap: wrap; }
|
||||
.tag-row [data-del] { grid-column: 4; grid-row: 1; }
|
||||
|
||||
Reference in new issue
Block a user