Kernel messages (dmesg) source read from /dev/kmsg

New source in Settings > Sources, off by default: reads the kernel ring
buffer of the host through /dev/kmsg, dates records from the boot time and
resumes after the last sequence number of the same boot (no duplicates on
restart). Missing device or denied access is shown in the UI, retried every
10 s. docker-compose.yml and .env.example document the device and the
kernel.dmesg_restrict / CAP_SYSLOG options.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
cedricandClaude Opus 5.5 committed 2026-10-04 09:52:17 +02:00
1 parent b626b3b8db
commit 0beaa24f82
12 files changed
+948 -4

No files matched your search

+7
View File
@@ -53,3 +53,10 @@ DOCKER_BACKFILL=1h
HOST_LOGS_GID=4 HOST_LOGS_GID=4
# History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new entries) # History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new entries)
HOST_LOGS_BACKFILL=1h HOST_LOGS_BACKFILL=1h
# Kernel messages of the host (dmesg, enable them in Settings > Sources). The container needs
# /dev/kmsg: uncomment "devices" in docker-compose.yml. When the host has
# kernel.dmesg_restrict=1 (default on Debian/Ubuntu), either run on the host
# sysctl -w kernel.dmesg_restrict=0 (persist it in /etc/sysctl.d/)
# or run the container as root with the SYSLOG capability ("user" and "cap_add" in docker-compose.yml).
# History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new messages)
DMESG_BACKFILL=1h
+40
View File
@@ -203,6 +203,43 @@ configurer sur l'hôte. La source est **désactivée par défaut** : activez-la
sont comptés dans les Paramètres et affichés comme « (compressed journal entry) ». Les fichiers sont comptés dans les Paramètres et affichés comme « (compressed journal entry) ». Les fichiers
texte tournés ou compressés (`*.1`, `*.gz`) ne sont pas lus. texte tournés ou compressés (`*.1`, `*.gz`) ne sont pas lus.
## Messages du noyau (dmesg)
Logstream peut collecter le buffer circulaire du noyau de l'hôte (ce qu'affiche `dmesg` :
matériel, pilotes, OOM killer, pare-feu…) en lisant `/dev/kmsg`. La source est **désactivée par
défaut** : activez-la dans **Paramètres > Sources > Messages du noyau (dmesg)** (enregistré dans
`/data/dmesg.json`).
- **Périphérique** : le conteneur ne voit pas `/dev/kmsg` par défaut. Décommentez la section
`devices` de `docker-compose.yml` (`/dev/kmsg:/dev/kmsg:r`, en lecture seule), puis
`docker compose up -d`.
- **Droits** : si l'hôte a `kernel.dmesg_restrict=1` (le défaut sur Debian et Ubuntu), seuls
les processus qui ont la capacité `CAP_SYSLOG` peuvent lire le log du noyau, et le conteneur
tourne avec un utilisateur sans privilège. Au choix :
- l'autoriser sur l'hôte : `sysctl -w kernel.dmesg_restrict=0`, conservé au redémarrage avec
`echo kernel.dmesg_restrict=0 | sudo tee /etc/sysctl.d/50-dmesg.conf` (tous les
utilisateurs de l'hôte peuvent alors lancer `dmesg`) ; ce réglage ne peut pas être changé
depuis un conteneur ;
- ou lancer Logstream en root avec la capacité `SYSLOG` : décommentez `user: "0:0"` et
`cap_add: [SYSLOG]` dans `docker-compose.yml`.
Paramètres > Sources indique si le périphérique est absent ou l'accès refusé ; le reste de
Logstream continue de fonctionner et le périphérique est réessayé toutes les 10 secondes.
- **Champs** : **app** vaut `kernel` (ou le nom du programme pour les messages écrits par des
programmes, comme `systemd[1]: …`), la sévérité et la facility viennent de l'enregistrement,
**host** vaut `localhost`, `subsystem` et `device` sont conservés quand le noyau les fournit.
Ces logs ont la source `kernel` (`proto` `kmsg`) : le filtre **Source** permet de les afficher
seuls.
- **Heure** : les enregistrements portent le temps écoulé depuis le démarrage ; Logstream le
convertit en date grâce à l'heure de démarrage (`/proc/uptime`). Comme avec `dmesg -T`,
l'heure peut être trop tôt après une mise en veille de l'hôte.
- À l'activation, la dernière heure est lue d'abord (`DMESG_BACKFILL`, `0` = seulement les
nouveaux messages). Le numéro de séquence du dernier message et l'identifiant de démarrage
sont enregistrés dans `/data/dmesg-state.json` : après un redémarrage de Logstream, la lecture
reprend après le dernier message (sans doublon) ; après un redémarrage de l'hôte, le nouveau
buffer est lu depuis `DMESG_BACKFILL`. Les messages écrasés dans le buffer du noyau avant
d'être lus sont comptés dans Paramètres.
## Export CSV ## Export CSV
Le bouton **Exporter** (à côté du nombre de logs) télécharge tous les logs stockés qui Le bouton **Exporter** (à côté du nombre de logs) télécharge tous les logs stockés qui
@@ -396,6 +433,8 @@ résolutions.
| `HOST_LOGS_GID` | `4` (adm) dans compose | groupe donné au conteneur pour lire les logs de l'hôte | | `HOST_LOGS_GID` | `4` (adm) dans compose | groupe donné au conteneur pour lire les logs de l'hôte |
| `HOST_LOGS_BACKFILL` | `1h` | historique lu à l'activation de la source « logs système de l'hôte » | | `HOST_LOGS_BACKFILL` | `1h` | historique lu à l'activation de la source « logs système de l'hôte » |
| `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte | | `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte |
| `DMESG_BACKFILL` | `1h` | historique lu à l'activation de la source « messages du noyau » |
| `DMESG_DEVICE` | `/dev/kmsg` | périphérique du log noyau lu par la source « messages du noyau » |
| `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) | | `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) |
| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion | | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion |
| `SPOOL_MAX_MB` | `1024` | taille du tampon disque des lots refusés par VictoriaLogs (`0` = pas de tampon : 15 s de tentatives, puis perte) | | `SPOOL_MAX_MB` | `1024` | taille du tampon disque des lots refusés par VictoriaLogs (`0` = pas de tampon : 15 s de tentatives, puis perte) |
@@ -456,6 +495,7 @@ Pour mettre à jour l'une d'elles :
| `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) | | `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
| `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources | | `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources |
| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` | | `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
| `dmesg.go` | messages du noyau : lecture de `/dev/kmsg`, analyse des enregistrements, position par identifiant de démarrage et numéro de séquence |
| `tags.go` | stockage des tags de couleur | | `tags.go` | stockage des tags de couleur |
| `presets.go`, `presets.json` | préréglages de tags (`/api/presets`), liste intégrée | | `presets.go`, `presets.json` | préréglages de tags (`/api/presets`), liste intégrée |
| `api.go` | routes HTTP `/api/*` | | `api.go` | routes HTTP `/api/*` |
+36
View File
@@ -187,6 +187,39 @@ configuring anything on the host. The source is **off by default**: turn it on i
shown as "(compressed journal entry)". Rotated or compressed text files (`*.1`, `*.gz`) are shown as "(compressed journal entry)". Rotated or compressed text files (`*.1`, `*.gz`) are
not read. not read.
## Kernel messages (dmesg)
Logstream can collect the kernel ring buffer of the host (what `dmesg` shows: hardware,
drivers, OOM killer, firewall…) by reading `/dev/kmsg`. The source is **off by default**: turn it
on in **Settings > Sources > Kernel messages (dmesg)** (saved in `/data/dmesg.json`).
- **Device**: the container does not see `/dev/kmsg` by default. Uncomment the `devices`
section of `docker-compose.yml` (`/dev/kmsg:/dev/kmsg:r`, read-only), then
`docker compose up -d`.
- **Permissions**: when the host has `kernel.dmesg_restrict=1` (the default on Debian and
Ubuntu), only processes with the `CAP_SYSLOG` capability can read the kernel log, and the
container runs as an unprivileged user. Either:
- allow it on the host: `sysctl -w kernel.dmesg_restrict=0`, kept across reboots with
`echo kernel.dmesg_restrict=0 | sudo tee /etc/sysctl.d/50-dmesg.conf` (every user of the
host can then run `dmesg`); this setting cannot be changed from a container;
- or run Logstream as root with the `SYSLOG` capability: uncomment `user: "0:0"` and
`cap_add: [SYSLOG]` in `docker-compose.yml`.
Settings > Sources says when the device is missing or access is denied; the rest of
Logstream keeps working and the device is tried again every 10 seconds.
- **Fields**: **app** is `kernel` (or the program name for messages written by programs such
as `systemd[1]: …`), severity and facility come from the record, **host** is `localhost`,
`subsystem` and `device` are kept when the kernel gives them. These logs have the `kernel`
source (`proto` `kmsg`): the **Source** filter shows them alone.
- **Time**: records carry the time since boot; Logstream turns it into a date using the boot
time (`/proc/uptime`). As with `dmesg -T`, the time can be early after the host was suspended.
- When the source is turned on, the last hour is read first (`DMESG_BACKFILL`, `0` = only new
messages). The sequence number of the last message and the boot ID are saved in
`/data/dmesg-state.json`: after a restart of Logstream, reading resumes after the last
message (no duplicates); after a reboot of the host, the new buffer is read from
`DMESG_BACKFILL` ago. Messages overwritten in the kernel buffer before being read are counted
in Settings.
## CSV export ## CSV export
The **Export** button (next to the log count) downloads every stored log matching the The **Export** button (next to the log count) downloads every stored log matching the
@@ -363,6 +396,8 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
| `HOST_LOGS_GID` | `4` (adm) in compose | group given to the container to read the host logs | | `HOST_LOGS_GID` | `4` (adm) in compose | group given to the container to read the host logs |
| `HOST_LOGS_BACKFILL` | `1h` | history read when the host system logs source is turned on | | `HOST_LOGS_BACKFILL` | `1h` | history read when the host system logs source is turned on |
| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted | | `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted |
| `DMESG_BACKFILL` | `1h` | history read when the kernel messages source is turned on |
| `DMESG_DEVICE` | `/dev/kmsg` | kernel log device read by the kernel messages source |
| `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) | | `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) |
| `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning | | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning |
| `SPOOL_MAX_MB` | `1024` | disk buffer size for batches VictoriaLogs could not take (`0` = no buffer: retried for 15 s, then dropped) | | `SPOOL_MAX_MB` | `1024` | disk buffer size for batches VictoriaLogs could not take (`0` = no buffer: retried for 15 s, then dropped) |
@@ -421,6 +456,7 @@ To update one of them:
| `docker.go` | Docker container logs (API, followers, positions, level detection) | | `docker.go` | Docker container logs (API, followers, positions, level detection) |
| `syslogserver.go` | syslog listeners opened and closed from Settings > Sources | | `syslogserver.go` | syslog listeners opened and closed from Settings > Sources |
| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower | | `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
| `dmesg.go` | kernel messages: `/dev/kmsg` reader, record parsing, position by boot ID and sequence number |
| `tags.go` | color tag storage | | `tags.go` | color tag storage |
| `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list | | `presets.go`, `presets.json` | color tag presets (`/api/presets`), built-in list |
| `api.go` | `/api/*` HTTP routes | | `api.go` | `/api/*` HTTP routes |
+23
View File
@@ -23,6 +23,7 @@ type API struct {
docker *DockerManager // nil when DOCKER_LOGS is off docker *DockerManager // nil when DOCKER_LOGS is off
syslog *SyslogServer syslog *SyslogServer
host *HostLogs host *HostLogs
dmesg *Dmesg
dbstats dbStatsCache dbstats dbStatsCache
} }
@@ -49,6 +50,28 @@ func (a *API) Routes(mux *http.ServeMux) {
mux.HandleFunc("PUT /api/docker", a.dockerConfigure) mux.HandleFunc("PUT /api/docker", a.dockerConfigure)
mux.HandleFunc("GET /api/hostlogs", a.hostLogsStatus) mux.HandleFunc("GET /api/hostlogs", a.hostLogsStatus)
mux.HandleFunc("PUT /api/hostlogs", a.hostLogsConfigure) mux.HandleFunc("PUT /api/hostlogs", a.hostLogsConfigure)
mux.HandleFunc("GET /api/dmesg", a.dmesgStatus)
mux.HandleFunc("PUT /api/dmesg", a.dmesgConfigure)
}
// GET /api/dmesg: state of the kernel messages source (Settings > Sources).
func (a *API) dmesgStatus(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, a.dmesg.Status())
}
// PUT /api/dmesg {"enabled": bool}
func (a *API) dmesgConfigure(w http.ResponseWriter, r *http.Request) {
var cfg dmesgConfig
if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&cfg); err != nil {
writeErr(w, http.StatusBadRequest, err)
return
}
if err := a.dmesg.Configure(cfg); err != nil {
writeErr(w, http.StatusInternalServerError, err)
return
}
log.Printf("kernel messages source changed from %s: %+v", r.RemoteAddr, cfg)
writeJSON(w, http.StatusOK, a.dmesg.Status())
} }
// GET /api/hostlogs: state of the host system logs source (Settings > Sources). // GET /api/hostlogs: state of the host system logs source (Settings > Sources).
+482
View File
@@ -0,0 +1,482 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"io/fs"
"log"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"syscall"
"time"
"unicode/utf8"
)
// dmesgConfig is saved in /data/dmesg.json and edited in Settings > Sources.
type dmesgConfig struct {
Enabled bool `json:"enabled"`
}
// dmesgPos is where reading resumes: the last record sent, valid for one boot
// only (the kernel numbers its records from 0 at each boot).
type dmesgPos struct {
BootID string `json:"boot_id"`
Seq uint64 `json:"seq"`
}
// Dmesg collects the kernel ring buffer of the machine hosting the stack by
// reading /dev/kmsg, the device dmesg itself reads.
type Dmesg struct {
device string // /dev/kmsg
bootIDPath string // identifies the current boot
uptimePath string // gives the boot time, to date the records
sink func(*Entry)
backfill time.Duration
cfgPath string
statePath string
retry time.Duration // delay before opening the device again after a failure
wake chan struct{}
mu sync.Mutex
cfg dmesgConfig
reset bool
pos dmesgPos
dirty bool
open bool
read uint64
lost uint64
errCode string
errDetail string
}
func NewDmesg(device, dataDir string, backfill time.Duration, sink func(*Entry)) *Dmesg {
d := &Dmesg{
device: device,
bootIDPath: "/proc/sys/kernel/random/boot_id",
uptimePath: "/proc/uptime",
sink: sink,
backfill: backfill,
cfgPath: filepath.Join(dataDir, "dmesg.json"),
statePath: filepath.Join(dataDir, "dmesg-state.json"),
retry: 10 * time.Second,
wake: make(chan struct{}, 1),
}
if b, err := os.ReadFile(d.cfgPath); err == nil {
_ = json.Unmarshal(b, &d.cfg)
}
if b, err := os.ReadFile(d.statePath); err == nil {
_ = json.Unmarshal(b, &d.pos)
}
return d
}
// kmsgSession is one opening of the device, read by its own goroutine.
type kmsgSession struct {
f *os.File
done chan struct{}
err error
stopped bool // closed on purpose: err is not a failure
}
func (s *kmsgSession) stop() {
s.stopped = true
s.f.Close() // ends a pending read (/dev/kmsg goes through the poller)
select {
case <-s.done:
case <-time.After(2 * time.Second):
}
}
// Run keeps the device open while the source is enabled.
func (d *Dmesg) Run(ctx context.Context) {
save := time.NewTicker(5 * time.Second)
defer save.Stop()
var cur *kmsgSession
var retryAt time.Time
for {
d.mu.Lock()
enabled, reset := d.cfg.Enabled, d.reset
d.reset = false
d.mu.Unlock()
if reset {
// Disabled then enabled again: start over from DMESG_BACKFILL ago
// rather than reading everything logged in between.
if cur != nil {
cur.stop()
cur = nil
}
d.mu.Lock()
d.pos, d.dirty = dmesgPos{}, true
d.mu.Unlock()
d.saveState()
retryAt = time.Time{}
}
if !enabled && cur != nil {
cur.stop()
cur = nil
d.mu.Lock()
d.open = false
d.mu.Unlock()
}
if enabled && cur == nil && !time.Now().Before(retryAt) {
var err error
if cur, err = d.start(); err != nil {
d.fail(err)
retryAt = time.Now().Add(d.retry)
}
}
var done chan struct{}
if cur != nil {
done = cur.done
}
wait := d.retry
if cur == nil && enabled {
wait = max(time.Until(retryAt), 10*time.Millisecond)
}
timer := time.NewTimer(wait)
select {
case <-ctx.Done():
timer.Stop()
if cur != nil {
cur.stop()
}
d.saveState()
return
case <-d.wake:
case <-save.C:
d.saveState()
case <-timer.C:
case <-done:
if !cur.stopped {
d.fail(cur.err)
}
cur = nil
retryAt = time.Now().Add(d.retry)
}
timer.Stop()
}
}
func (d *Dmesg) saveState() {
d.mu.Lock()
if !d.dirty {
d.mu.Unlock()
return
}
d.dirty = false
pos := d.pos
d.mu.Unlock()
if err := writeJSONFile(d.statePath, pos); err != nil {
log.Printf("dmesg: saving position: %v", err)
}
}
// fail records why the device is not being read (nil error: end of a
// regular file, only seen in tests).
func (d *Dmesg) fail(err error) {
code, detail := "", ""
if err != nil && !errors.Is(err, io.EOF) {
detail = err.Error()
switch {
case errors.Is(err, fs.ErrNotExist):
code = "not_found"
case errors.Is(err, fs.ErrPermission):
code = "permission"
default:
code = "read"
}
}
d.mu.Lock()
changed := code != d.errCode
d.open, d.errCode, d.errDetail = false, code, detail
d.mu.Unlock()
if changed && code != "" {
log.Printf("dmesg: %v", err)
}
}
// start opens the device and starts reading it in the background.
func (d *Dmesg) start() (*kmsgSession, error) {
f, err := os.Open(d.device)
if err != nil {
return nil, err
}
bootID := readTrim(d.bootIDPath)
boot := d.bootTime(time.Now())
d.mu.Lock()
pos := d.pos
d.open, d.errCode, d.errDetail = true, "", ""
d.mu.Unlock()
// Same boot as the saved position: skip what was already sent. Otherwise
// read from DMESG_BACKFILL ago (0: only new records).
resume := pos.BootID != "" && pos.BootID == bootID
var notBefore time.Time
if !resume {
if d.backfill <= 0 {
if _, err := f.Seek(0, io.SeekEnd); err != nil {
f.Close()
return nil, err
}
} else if !boot.IsZero() {
notBefore = time.Now().Add(-d.backfill)
}
d.mu.Lock()
d.pos, d.dirty = dmesgPos{BootID: bootID}, true
d.mu.Unlock()
}
s := &kmsgSession{f: f, done: make(chan struct{})}
go func() {
defer close(s.done)
s.err = d.consume(f, bootID, boot, resume, pos.Seq, notBefore)
}()
return s, nil
}
func readTrim(path string) string {
b, err := os.ReadFile(path)
if err != nil {
return ""
}
return strings.TrimSpace(string(b))
}
// bootTime is now minus the uptime; zero when unknown (records are then dated
// when they are read).
func (d *Dmesg) bootTime(now time.Time) time.Time {
f := strings.Fields(readTrim(d.uptimePath))
if len(f) == 0 {
return time.Time{}
}
up, err := strconv.ParseFloat(f[0], 64)
if err != nil || up <= 0 {
return time.Time{}
}
return now.Add(-time.Duration(up * float64(time.Second)))
}
// consume reads records until an error. Each read of /dev/kmsg returns one
// record; several records in one read (a regular file, in tests) are split.
func (d *Dmesg) consume(r io.Reader, bootID string, boot time.Time, resume bool, after uint64, notBefore time.Time) error {
buf := make([]byte, 64<<10) // a record is at most 8 KiB
last, haveLast := after, resume
for {
n, err := r.Read(buf)
if n > 0 {
for _, raw := range splitKmsg(buf[:n]) {
rec, ok := parseKmsg(raw)
if !ok {
continue
}
if haveLast && rec.Seq <= last {
continue // already sent before a restart
}
var lost uint64
if haveLast && rec.Seq > last+1 {
lost = rec.Seq - last - 1 // overwritten in the ring buffer before being read
}
last, haveLast = rec.Seq, true
now := time.Now()
t := now
if !boot.IsZero() {
t = boot.Add(time.Duration(rec.Usec) * time.Microsecond)
}
if !notBefore.IsZero() && t.Before(notBefore) {
d.advance(bootID, rec.Seq, 0, 0)
continue
}
d.sink(kmsgEntry(rec, t, now))
d.advance(bootID, rec.Seq, 1, lost)
}
}
if err != nil {
if errors.Is(err, os.ErrClosed) {
return nil
}
if errors.Is(err, syscall.EPIPE) {
continue // records overwritten while reading: the next read goes on
}
return err
}
}
}
func (d *Dmesg) advance(bootID string, seq, read, lost uint64) {
d.mu.Lock()
d.pos, d.dirty = dmesgPos{BootID: bootID, Seq: seq}, true
d.read += read
d.lost += lost
d.mu.Unlock()
}
// kmsgRecord is one record of /dev/kmsg:
//
// prio,seq,usec,flags[,...];message
// KEY=value (optional dictionary lines)
type kmsgRecord struct {
Prio int
Seq uint64
Usec uint64
Msg string
Dict map[string]string
}
// splitKmsg cuts a buffer into records: a new record starts after a newline
// that is not followed by a space (dictionary lines start with one).
func splitKmsg(b []byte) [][]byte {
var out [][]byte
start := 0
for i := 0; i < len(b); i++ {
if b[i] == '\n' && (i+1 == len(b) || b[i+1] != ' ') {
out = append(out, b[start:i])
start = i + 1
}
}
if start < len(b) {
out = append(out, b[start:])
}
return out
}
func parseKmsg(raw []byte) (kmsgRecord, bool) {
var rec kmsgRecord
lines := bytes.Split(raw, []byte{'\n'})
head, msg, ok := bytes.Cut(lines[0], []byte{';'})
if !ok {
return rec, false
}
f := strings.Split(string(head), ",")
if len(f) < 3 {
return rec, false
}
prio, err1 := strconv.Atoi(f[0])
seq, err2 := strconv.ParseUint(f[1], 10, 64)
usec, err3 := strconv.ParseUint(f[2], 10, 64)
if err1 != nil || err2 != nil || err3 != nil || prio < 0 {
return rec, false
}
rec.Prio, rec.Seq, rec.Usec = prio, seq, usec
rec.Msg = unescapeKmsg(string(msg))
for _, l := range lines[1:] {
if k, v, ok := strings.Cut(strings.TrimPrefix(string(l), " "), "="); ok && k != "" {
if rec.Dict == nil {
rec.Dict = map[string]string{}
}
rec.Dict[k] = unescapeKmsg(v)
}
}
return rec, true
}
// unescapeKmsg decodes the \xNN escapes the kernel uses for non-printable bytes.
func unescapeKmsg(s string) string {
if !strings.Contains(s, `\x`) {
return s
}
var b strings.Builder
for i := 0; i < len(s); i++ {
if s[i] == '\\' && i+3 < len(s) && s[i+1] == 'x' {
if v, err := strconv.ParseUint(s[i+2:i+4], 16, 8); err == nil {
b.WriteByte(byte(v))
i += 3
continue
}
}
b.WriteByte(s[i])
}
return b.String()
}
// kmsgEntry turns a record into an Entry. Kernel records have the kern
// facility; programs writing to /dev/kmsg (systemd at boot…) use another
// one and prefix their name: "systemd[1]: …".
func kmsgEntry(rec kmsgRecord, t, now time.Time) *Entry {
sev := rec.Prio & 7
fac := rec.Prio >> 3
if fac >= len(facilityNames) {
fac = 1 // user
}
msg := strings.TrimRight(rec.Msg, "\n")
if !utf8.ValidString(msg) {
msg = strings.ToValidUTF8(msg, "�")
}
app, pid := "kernel", ""
if fac != 0 {
if a, p, rest, ok := splitTag(msg); ok {
app, pid, msg = a, p, rest
}
}
e := &Entry{
Time: t,
Received: now,
Host: "localhost",
App: app,
ProcID: pid,
Facility: facilityNames[fac],
Severity: severityNames[sev],
SevNum: sev,
Message: msg,
Proto: "kmsg",
SourceType: "kernel",
Wait: true,
}
if s, dev := rec.Dict["SUBSYSTEM"], rec.Dict["DEVICE"]; s != "" || dev != "" {
e.Extra = map[string]string{}
if s != "" {
e.Extra["subsystem"] = s
}
if dev != "" {
e.Extra["device"] = dev
}
}
return e
}
// splitTag cuts "name[pid]: message" or "name: message".
func splitTag(msg string) (app, pid, rest string, ok bool) {
tag, rest, found := strings.Cut(msg, ": ")
if !found || tag == "" || len(tag) > 48 || strings.ContainsAny(tag, " \t") {
return "", "", "", false
}
if i := strings.IndexByte(tag, '['); i > 0 && strings.HasSuffix(tag, "]") {
return tag[:i], tag[i+1 : len(tag)-1], rest, true
}
return tag, "", rest, true
}
// Configure saves and applies the configuration.
func (d *Dmesg) Configure(cfg dmesgConfig) error {
d.mu.Lock()
if d.cfg.Enabled && !cfg.Enabled {
d.reset = true
d.errCode, d.errDetail = "", ""
}
d.cfg = cfg
d.mu.Unlock()
select {
case d.wake <- struct{}{}:
default:
}
return writeJSONFile(d.cfgPath, cfg)
}
// Status is the state shown in Settings > Sources.
func (d *Dmesg) Status() map[string]any {
d.mu.Lock()
defer d.mu.Unlock()
return map[string]any{
"enabled": d.cfg.Enabled,
"open": d.open && d.cfg.Enabled,
"device": d.device,
"read": d.read,
"lost": d.lost,
"code": d.errCode,
"error": d.errDetail,
}
}
+265
View File
@@ -0,0 +1,265 @@
package main
import (
"context"
"os"
"path/filepath"
"strconv"
"strings"
"sync"
"testing"
"time"
)
func TestParseKmsg(t *testing.T) {
rec, ok := parseKmsg([]byte("6,339,5140900,-;NET: Registered protocol family 10\n SUBSYSTEM=net\n DEVICE=n2"))
if !ok || rec.Prio != 6 || rec.Seq != 339 || rec.Usec != 5140900 || rec.Msg != "NET: Registered protocol family 10" {
t.Fatalf("record %+v", rec)
}
if rec.Dict["SUBSYSTEM"] != "net" || rec.Dict["DEVICE"] != "n2" {
t.Fatalf("dict %+v", rec.Dict)
}
// Extra header fields (newer kernels: caller id) and a ';' in the message.
rec, ok = parseKmsg([]byte("4,12,99,c,caller=T1;a;b \\x1b[31mred\\x5c"))
if !ok || rec.Seq != 12 || rec.Msg != "a;b \x1b[31mred\\" {
t.Fatalf("record %+v", rec)
}
for _, bad := range []string{"", "no header", "6,1;short", "x,1,2,-;msg", "6,-1,2,-;msg", "-1,1,2,-;msg"} {
if _, ok := parseKmsg([]byte(bad)); ok {
t.Errorf("%q parsed", bad)
}
}
}
func TestUnescapeKmsg(t *testing.T) {
for in, want := range map[string]string{
`plain`: "plain",
`a\x0ab`: "a\nb",
`end\x4`: `end\x4`,
`bad\xzz`: `bad\xzz`,
`\x41\x42\x4`: `AB\x4`,
} {
if got := unescapeKmsg(in); got != want {
t.Errorf("%q: got %q, want %q", in, got, want)
}
}
}
func TestSplitKmsg(t *testing.T) {
got := splitKmsg([]byte("6,1,0,-;one\n SUBSYSTEM=pci\n6,2,0,-;two\n6,3,0,-;three"))
if len(got) != 3 || string(got[0]) != "6,1,0,-;one\n SUBSYSTEM=pci" || string(got[2]) != "6,3,0,-;three" {
t.Fatalf("got %q", got)
}
}
func TestKmsgEntry(t *testing.T) {
boot := time.Date(2026, 10, 4, 6, 0, 0, 0, time.UTC)
now := boot.Add(time.Hour)
rec, _ := parseKmsg([]byte("3,7,2500000,-;usb 1-1: device descriptor read/64, error -71\n SUBSYSTEM=usb\n DEVICE=c189:1"))
e := kmsgEntry(rec, boot.Add(2500*time.Millisecond), now)
if e.App != "kernel" || e.Facility != "kern" || e.Severity != "err" || e.SevNum != 3 || e.SourceType != "kernel" || e.Proto != "kmsg" || !e.Wait {
t.Fatalf("entry %+v", e)
}
if e.Message != "usb 1-1: device descriptor read/64, error -71" || e.Extra["subsystem"] != "usb" || e.Extra["device"] != "c189:1" {
t.Fatalf("entry %+v", e)
}
if !e.Time.Equal(boot.Add(2500*time.Millisecond)) || !e.Received.Equal(now) {
t.Fatalf("times %v %v", e.Time, e.Received)
}
// Written by a program (facility daemon = 3<<3 | info).
rec, _ = parseKmsg([]byte("30,8,3000000,-;systemd[1]: Started Journal Service."))
e = kmsgEntry(rec, now, now)
if e.App != "systemd" || e.ProcID != "1" || e.Facility != "daemon" || e.Severity != "info" || e.Message != "Started Journal Service." || e.Extra != nil {
t.Fatalf("entry %+v", e)
}
// Facility out of range.
rec, _ = parseKmsg([]byte("255,9,0,-;odd"))
if e = kmsgEntry(rec, now, now); e.Facility != "user" || e.Severity != "debug" {
t.Fatalf("entry %+v", e)
}
}
// fakeHost writes a kmsg file, a boot id and an uptime in a temp dir.
func fakeHost(t *testing.T, uptime float64, records ...string) (*Dmesg, *[]*Entry, *sync.Mutex) {
t.Helper()
dir := t.TempDir()
write := func(name, s string) string {
p := filepath.Join(dir, name)
if err := os.WriteFile(p, []byte(s), 0o644); err != nil {
t.Fatal(err)
}
return p
}
var mu sync.Mutex
var got []*Entry
d := NewDmesg(write("kmsg", strings.Join(records, "\n")+"\n"), filepath.Join(dir, "data"), time.Hour, func(e *Entry) {
mu.Lock()
got = append(got, e)
mu.Unlock()
})
d.bootIDPath = write("boot_id", "b1\n")
d.uptimePath = write("uptime", strconv.FormatFloat(uptime, 'f', 2, 64)+" 1234.00\n")
d.retry = 20 * time.Millisecond
return d, &got, &mu
}
func TestDmesgBackfillAndResume(t *testing.T) {
// Booted 3 h ago: the record at 1 s is outside the 1 h backfill window,
// the ones at 2.5 h and 2.9 h are inside.
d, got, mu := fakeHost(t, 3*3600,
"6,1,1000000,-;Linux version 6.12",
"4,2,9000000000,-;eth0: link down",
"6,3,10440000000,-;eth0: link up",
)
s, err := d.start()
if err != nil {
t.Fatal(err)
}
<-s.done
mu.Lock()
if len(*got) != 2 || (*got)[0].Message != "eth0: link down" || (*got)[1].Message != "eth0: link up" {
t.Fatalf("got %+v", *got)
}
if age := time.Since((*got)[1].Time); age < 5*time.Minute || age > 7*time.Minute {
t.Fatalf("record dated %v ago", age)
}
mu.Unlock()
d.saveState()
// Restart with the saved position: same boot, nothing is sent again,
// and a new record is.
f, _ := os.OpenFile(d.device, os.O_APPEND|os.O_WRONLY, 0)
_, _ = f.WriteString("6,6,10500000000,-;eth0: renamed\n")
f.Close()
d2 := NewDmesg(d.device, filepath.Dir(d.statePath), time.Hour, d.sink)
d2.bootIDPath, d2.uptimePath = d.bootIDPath, d.uptimePath
if d2.pos != (dmesgPos{BootID: "b1", Seq: 3}) {
t.Fatalf("saved position %+v", d2.pos)
}
s, err = d2.start()
if err != nil {
t.Fatal(err)
}
<-s.done
mu.Lock()
defer mu.Unlock()
if len(*got) != 3 || (*got)[2].Message != "eth0: renamed" || (*got)[2].App != "kernel" {
t.Fatalf("after restart %+v", (*got)[2:])
}
if st := d2.Status(); st["lost"] != uint64(2) || st["read"] != uint64(1) {
t.Fatalf("status %+v", st) // records 4 and 5 were overwritten
}
}
func TestDmesgNewBoot(t *testing.T) {
d, got, mu := fakeHost(t, 60, "6,1,1000000,-;booted")
d.pos = dmesgPos{BootID: "old", Seq: 500}
s, err := d.start()
if err != nil {
t.Fatal(err)
}
<-s.done
mu.Lock()
defer mu.Unlock()
if len(*got) != 1 || d.pos != (dmesgPos{BootID: "b1", Seq: 1}) {
t.Fatalf("got %d entries, position %+v", len(*got), d.pos)
}
}
func TestDmesgNoBackfill(t *testing.T) {
d, got, mu := fakeHost(t, 60, "6,1,1000000,-;old")
d.backfill = 0
s, err := d.start()
if err != nil {
t.Fatal(err)
}
<-s.done
mu.Lock()
defer mu.Unlock()
if len(*got) != 0 {
t.Fatalf("got %+v", *got)
}
}
func TestDmesgRun(t *testing.T) {
d, got, mu := fakeHost(t, 60, "6,1,1000000,-;first", "6,2,2000000,-;second")
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
d.Run(ctx)
close(done)
}()
if err := d.Configure(dmesgConfig{Enabled: true}); err != nil {
t.Fatal(err)
}
deadline := time.Now().Add(5 * time.Second)
for {
mu.Lock()
n := len(*got)
mu.Unlock()
if n >= 2 {
break
}
if time.Now().After(deadline) {
t.Fatalf("got %d entries", n)
}
time.Sleep(10 * time.Millisecond)
}
// The file is re-opened after each end of file: still no duplicate.
time.Sleep(100 * time.Millisecond)
cancel()
<-done
mu.Lock()
defer mu.Unlock()
if len(*got) != 2 {
t.Fatalf("got %d entries", len(*got))
}
b, err := os.ReadFile(d.statePath)
if err != nil || !strings.Contains(string(b), `"seq": 2`) {
t.Fatalf("state %s %v", b, err)
}
b, _ = os.ReadFile(d.cfgPath)
if !strings.Contains(string(b), `"enabled": true`) {
t.Fatalf("config %s", b)
}
}
func TestDmesgMissingDevice(t *testing.T) {
d := NewDmesg(filepath.Join(t.TempDir(), "kmsg"), t.TempDir(), time.Hour, func(*Entry) {})
d.retry = 20 * time.Millisecond
ctx, cancel := context.WithCancel(context.Background())
done := make(chan struct{})
go func() {
d.Run(ctx)
close(done)
}()
_ = d.Configure(dmesgConfig{Enabled: true})
deadline := time.Now().Add(5 * time.Second)
for d.Status()["code"] != "not_found" {
if time.Now().After(deadline) {
t.Fatalf("status %+v", d.Status())
}
time.Sleep(10 * time.Millisecond)
}
// Turning it off clears the error.
_ = d.Configure(dmesgConfig{Enabled: false})
if st := d.Status(); st["code"] != "" || st["enabled"] != false {
t.Fatalf("status %+v", st)
}
cancel()
<-done
}
func TestDmesgPermission(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("root reads everything")
}
p := filepath.Join(t.TempDir(), "kmsg")
_ = os.WriteFile(p, []byte("6,1,0,-;x\n"), 0o000)
d := NewDmesg(p, t.TempDir(), time.Hour, func(*Entry) {})
_, err := d.start()
d.fail(err)
if st := d.Status(); st["code"] != "permission" {
t.Fatalf("status %+v", st)
}
}
+11
View File
@@ -37,8 +37,19 @@ services:
DOCKER_HOST: tcp://docker-proxy:2375 # lecture seul de l'API Docker DOCKER_HOST: tcp://docker-proxy:2375 # lecture seul de l'API Docker
DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h} DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h}
HOST_LOGS_BACKFILL: ${HOST_LOGS_BACKFILL:-1h} # historique lu a l'activation des logs systeme de l'hote HOST_LOGS_BACKFILL: ${HOST_LOGS_BACKFILL:-1h} # historique lu a l'activation des logs systeme de l'hote
DMESG_BACKFILL: ${DMESG_BACKFILL:-1h} # historique lu a l'activation des messages du noyau (dmesg)
group_add: group_add:
- "${HOST_LOGS_GID:-4}" # groupe autorise a lire les logs de l'hote (4 = adm sur Debian/Ubuntu) - "${HOST_LOGS_GID:-4}" # groupe autorise a lire les logs de l'hote (4 = adm sur Debian/Ubuntu)
# messages du noyau de l'hote (dmesg), source a activer dans Reglages > Sources :
# decommenter devices pour donner /dev/kmsg (lecture seule) au conteneur
# devices:
# - /dev/kmsg:/dev/kmsg:r
# si l'hote a kernel.dmesg_restrict=1 (defaut Debian/Ubuntu), soit sur l'hote :
# sysctl -w kernel.dmesg_restrict=0 (et kernel.dmesg_restrict = 0 dans /etc/sysctl.d/ pour le garder)
# soit lancer le conteneur en root avec la capacite SYSLOG (decommenter les deux) :
# user: "0:0"
# cap_add:
# - SYSLOG
volumes: volumes:
- logstream-data:/data # tags.json, docker.json (les choix des conteneurs) - logstream-data:/data # tags.json, docker.json (les choix des conteneurs)
# logs systeme de l'hote, en lecture seule (source a activer dans Reglages > Sources) # logs systeme de l'hote, en lecture seule (source a activer dans Reglages > Sources)
+7
View File
@@ -38,6 +38,8 @@ type config struct {
backfill time.Duration backfill time.Duration
hostRoot string hostRoot string
hostFill time.Duration hostFill time.Duration
kmsgDevice string
kmsgFill time.Duration
batchSize int batchSize int
queueSize int queueSize int
flushEvery time.Duration flushEvery time.Duration
@@ -115,6 +117,8 @@ func main() {
backfill: getenvDuration("DOCKER_BACKFILL", time.Hour), backfill: getenvDuration("DOCKER_BACKFILL", time.Hour),
hostRoot: getenv("HOST_LOGS_ROOT", "/host"), hostRoot: getenv("HOST_LOGS_ROOT", "/host"),
hostFill: getenvDuration("HOST_LOGS_BACKFILL", time.Hour), hostFill: getenvDuration("HOST_LOGS_BACKFILL", time.Hour),
kmsgDevice: getenv("DMESG_DEVICE", "/dev/kmsg"),
kmsgFill: getenvDuration("DMESG_BACKFILL", time.Hour),
batchSize: getenvInt("BATCH_SIZE", 1000), batchSize: getenvInt("BATCH_SIZE", 1000),
queueSize: getenvInt("QUEUE_SIZE", 100000), queueSize: getenvInt("QUEUE_SIZE", 100000),
flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond, flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond,
@@ -186,6 +190,9 @@ func main() {
// System logs of the host (journal or /var/log), off until enabled in Settings > Sources. // System logs of the host (journal or /var/log), off until enabled in Settings > Sources.
api.host = NewHostLogs(cfg.hostRoot, cfg.dataDir, cfg.hostFill, sink) api.host = NewHostLogs(cfg.hostRoot, cfg.dataDir, cfg.hostFill, sink)
go api.host.Run(ctx) go api.host.Run(ctx)
// Kernel messages of the host (/dev/kmsg), off until enabled in Settings > Sources.
api.dmesg = NewDmesg(cfg.kmsgDevice, cfg.dataDir, cfg.kmsgFill, sink)
go api.dmesg.Run(ctx)
api.Routes(mux) api.Routes(mux)
mux.Handle("GET /", http.FileServer(http.FS(static))) mux.Handle("GET /", http.FileServer(http.FS(static)))
+2 -2
View File
@@ -119,10 +119,10 @@ func (f Filter) filterExpr() string {
parts = append(parts, "app:="+strconv.Quote(f.App)) parts = append(parts, "app:="+strconv.Quote(f.App))
} }
switch f.Source { switch f.Source {
case "docker", "host": case "docker", "host", "kernel":
parts = append(parts, `source_type:=`+strconv.Quote(f.Source)) parts = append(parts, `source_type:=`+strconv.Quote(f.Source))
case "syslog": // also matches logs stored before source_type existed case "syslog": // also matches logs stored before source_type existed
parts = append(parts, `!(source_type:in("docker","host"))`) parts = append(parts, `!(source_type:in("docker","host","kernel"))`)
} }
if f.Severity >= 0 && f.Severity < 7 { if f.Severity >= 0 && f.Severity < 7 {
names := make([]string, 0, 8) names := make([]string, 0, 8)
+1 -1
View File
@@ -36,7 +36,7 @@ type Entry struct {
Proto string // udp or tcp Proto string // udp or tcp
HostIP string // original host value when it was an IP resolved through DNS HostIP string // original host value when it was an IP resolved through DNS
SourceType string // "syslog" or "docker" SourceType string // "syslog", "docker", "host" or "kernel"
Extra map[string]string // additional fields (docker: container, image, …) Extra map[string]string // additional fields (docker: container, image, …)
// Not stored. Wait: the producer can be slowed down when the queue is full // Not stored. Wait: the producer can be slowed down when the queue is full
+63 -1
View File
@@ -77,6 +77,7 @@ const I18N = {
clickHost: 'Click to filter on this host', clickApp: 'Click to filter on this app', clickHost: 'Click to filter on this host', clickApp: 'Click to filter on this app',
sourceAria: 'Source', srcAll: 'All sources', tabSources: 'Sources', sourceAria: 'Source', srcAll: 'All sources', tabSources: 'Sources',
srcHost: 'Host system', srcHost: 'Host system',
srcKernel: 'Kernel (dmesg)',
hostTitle: 'Host system logs', hostTitle: 'Host system logs',
hostEnabled: 'Collect the system logs of this machine', hostEnabled: 'Collect the system logs of this machine',
hostOff: 'Off: the system logs of the machine hosting LogStream are not collected.', hostOff: 'Off: the system logs of the machine hosting LogStream are not collected.',
@@ -89,6 +90,15 @@ const I18N = {
host_empty: 'No systemd journal and no log file found in /var/log.', host_empty: 'No systemd journal and no log file found in /var/log.',
host_read: 'Cannot read the host logs: ', host_read: 'Cannot read the host logs: ',
hostHelp: 'Reads the systemd journal of the host (/var/log/journal, /run/log/journal), or the text files of /var/log (syslog, messages, *.log) when there is no journal. Directories are mounted read-only. When turned on, the last hour is read first (HOST_LOGS_BACKFILL).', hostHelp: 'Reads the systemd journal of the host (/var/log/journal, /run/log/journal), or the text files of /var/log (syslog, messages, *.log) when there is no journal. Directories are mounted read-only. When turned on, the last hour is read first (HOST_LOGS_BACKFILL).',
dmesgTitle: 'Kernel messages (dmesg)',
dmesgEnabled: 'Collect the kernel messages of this machine',
dmesgOff: 'Off: the kernel messages of the machine hosting LogStream are not collected.',
dmesgOk: ({ d, r }) => `Reading ${d}: ${r} messages since startup.`,
dmesgLost: (n) => ` ${n} messages were overwritten in the kernel buffer before being read.`,
dmesg_not_found: 'Device /dev/kmsg not found in the container: uncomment its devices section in docker-compose.yml (see the README).',
dmesg_permission: 'Permission denied on /dev/kmsg: set kernel.dmesg_restrict=0 on the host, or run the container as root with the SYSLOG capability (see the README). ',
dmesg_read: 'Cannot read the kernel messages: ',
dmesgHelp: 'Reads the kernel ring buffer of the host through /dev/kmsg, like dmesg: hardware, drivers, OOM killer, firewall… When turned on, the last hour is read first (DMESG_BACKFILL); after a restart, reading resumes where it stopped.',
syslogEnabled: 'Receive syslog messages', syslogProtocols: 'Protocols', syslogPort: 'Port', syslogEnabled: 'Receive syslog messages', syslogProtocols: 'Protocols', syslogPort: 'Port',
syslogListening: ({ port, protos }) => `Listening on port ${port} (${protos}).`, syslogListening: ({ port, protos }) => `Listening on port ${port} (${protos}).`,
syslogOff: 'Syslog reception is off: syslog messages are ignored (Docker logs are still collected).', syslogOff: 'Syslog reception is off: syslog messages are ignored (Docker logs are still collected).',
@@ -251,6 +261,7 @@ const I18N = {
clickHost: 'Cliquer pour filtrer sur cet hôte', clickApp: 'Cliquer pour filtrer sur cette appli', clickHost: 'Cliquer pour filtrer sur cet hôte', clickApp: 'Cliquer pour filtrer sur cette appli',
sourceAria: 'Source', srcAll: 'Toutes les sources', tabSources: 'Sources', sourceAria: 'Source', srcAll: 'Toutes les sources', tabSources: 'Sources',
srcHost: 'Système hôte', srcHost: 'Système hôte',
srcKernel: 'Noyau (dmesg)',
hostTitle: 'Logs système de l\'hôte', hostTitle: 'Logs système de l\'hôte',
hostEnabled: 'Collecter les logs système de cette machine', hostEnabled: 'Collecter les logs système de cette machine',
hostOff: 'Désactivé : les logs système de la machine qui héberge LogStream ne sont pas collectés.', hostOff: 'Désactivé : les logs système de la machine qui héberge LogStream ne sont pas collectés.',
@@ -263,6 +274,15 @@ const I18N = {
host_empty: 'Ni journal systemd ni fichier de log trouvé dans /var/log.', host_empty: 'Ni journal systemd ni fichier de log trouvé dans /var/log.',
host_read: 'Lecture des logs de l\'hôte impossible : ', host_read: 'Lecture des logs de l\'hôte impossible : ',
hostHelp: 'Lit le journal systemd de l\'hôte (/var/log/journal, /run/log/journal), ou les fichiers texte de /var/log (syslog, messages, *.log) s\'il n\'y a pas de journal. Les répertoires sont montés en lecture seule. À l\'activation, la dernière heure est lue d\'abord (HOST_LOGS_BACKFILL).', hostHelp: 'Lit le journal systemd de l\'hôte (/var/log/journal, /run/log/journal), ou les fichiers texte de /var/log (syslog, messages, *.log) s\'il n\'y a pas de journal. Les répertoires sont montés en lecture seule. À l\'activation, la dernière heure est lue d\'abord (HOST_LOGS_BACKFILL).',
dmesgTitle: 'Messages du noyau (dmesg)',
dmesgEnabled: 'Collecter les messages du noyau de cette machine',
dmesgOff: 'Désactivé : les messages du noyau de la machine qui héberge LogStream ne sont pas collectés.',
dmesgOk: ({ d, r }) => `Lecture de ${d} : ${r} messages depuis le démarrage.`,
dmesgLost: (n) => ` ${n} messages ont été écrasés dans le buffer du noyau avant d'être lus.`,
dmesg_not_found: 'Périphérique /dev/kmsg absent du conteneur : décommentez sa section devices dans docker-compose.yml (voir le README).',
dmesg_permission: 'Accès refusé à /dev/kmsg : réglez kernel.dmesg_restrict=0 sur l\'hôte, ou lancez le conteneur en root avec la capacité SYSLOG (voir le README). ',
dmesg_read: 'Lecture des messages du noyau impossible : ',
dmesgHelp: 'Lit le buffer circulaire du noyau de l\'hôte via /dev/kmsg, comme dmesg : matériel, pilotes, OOM killer, pare-feu… À l\'activation, la dernière heure est lue d\'abord (DMESG_BACKFILL) ; après un redémarrage, la lecture reprend où elle s\'était arrêtée.',
syslogEnabled: 'Recevoir les messages syslog', syslogProtocols: 'Protocoles', syslogPort: 'Port', syslogEnabled: 'Recevoir les messages syslog', syslogProtocols: 'Protocoles', syslogPort: 'Port',
syslogListening: ({ port, protos }) => `Écoute sur le port ${port} (${protos}).`, syslogListening: ({ port, protos }) => `Écoute sur le port ${port} (${protos}).`,
syslogOff: 'Réception syslog désactivée : les messages syslog sont ignorés (les logs Docker sont toujours collectés).', syslogOff: 'Réception syslog désactivée : les messages syslog sont ignorés (les logs Docker sont toujours collectés).',
@@ -543,6 +563,7 @@ function setLang(next) {
renderInterface(); renderInterface();
renderSyslog(); renderSyslog();
renderHost(); renderHost();
renderDmesg();
renderDocker(); renderDocker();
} }
@@ -958,7 +979,7 @@ function updateCount(tookMs) {
} }
function detailsHTML(r) { function detailsHTML(r) {
const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'unit', 'log_file', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg']; const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'unit', 'log_file', 'subsystem', 'device', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg'];
const hidden = new Set(['_stream_id', '_stream', 'sevnum']); const hidden = new Set(['_stream_id', '_stream', 'sevnum']);
if (r.msg_time) hidden.add('_time'); // same as the reception time if (r.msg_time) hidden.add('_time'); // same as the reception time
const keys = order.filter((k) => r[k] != null && r[k] !== '' && !hidden.has(k)) const keys = order.filter((k) => r[k] != null && r[k] !== '' && !hidden.has(k))
@@ -2000,6 +2021,45 @@ async function configureHost(enabled) {
$('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked)); $('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked));
/* ================= Kernel messages (dmesg) ================= */
let dmesgState = null;
async function loadDmesg() {
try { dmesgState = await api('/api/dmesg'); } catch { dmesgState = null; }
renderDmesg();
}
function renderDmesg() {
const s = dmesgState;
const status = $('#dmesgStatus');
if (!s) { status.textContent = ''; return; }
$('#dmesgEnabled').checked = s.enabled;
let text = t('hostWaiting');
let cls = 'muted';
if (!s.enabled) {
text = t('dmesgOff');
} else if (s.code) {
text = t('dmesg_' + s.code) + (s.code === 'read' || s.code === 'permission' ? s.error || '' : '');
cls = 'bad';
} else if (s.open) {
text = t('dmesgOk', { d: s.device, r: fmtNum(s.read) }) + (s.lost ? t('dmesgLost', fmtNum(s.lost)) : '');
cls = 'good';
}
status.textContent = text;
status.className = 'docker-status ' + cls;
}
async function configureDmesg(enabled) {
dmesgState = { ...(dmesgState || {}), enabled };
renderDmesg();
try { dmesgState = await api('/api/dmesg', { method: 'PUT', body: { enabled } }); } catch (e) { toast(e.message); }
renderDmesg();
setTimeout(loadDmesg, 1500); // the device is opened in the background
}
$('#dmesgEnabled').addEventListener('change', (ev) => configureDmesg(ev.target.checked));
/* ================= Docker source ================= */ /* ================= Docker source ================= */
let dockerState = null; let dockerState = null;
@@ -2116,6 +2176,7 @@ setInterval(() => {
if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) { if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) {
loadDocker(); loadDocker();
loadHost(); loadHost();
loadDmesg();
} }
}, 4000); }, 4000);
@@ -2409,6 +2470,7 @@ $('#settingsBtn').addEventListener('click', () => {
loadPurgeStatus(); loadPurgeStatus();
loadSyslog(); loadSyslog();
loadHost(); loadHost();
loadDmesg();
loadDocker(); loadDocker();
renderDbStats(); renderDbStats();
loadDbStats(); loadDbStats();
+11
View File
@@ -102,6 +102,7 @@
<option value="syslog">Syslog</option> <option value="syslog">Syslog</option>
<option value="docker">Docker</option> <option value="docker">Docker</option>
<option value="host" data-i18n="srcHost">Host system</option> <option value="host" data-i18n="srcHost">Host system</option>
<option value="kernel" data-i18n="srcKernel">Kernel (dmesg)</option>
</select> </select>
<span class="spacer"></span> <span class="spacer"></span>
<span id="count" class="muted"></span> <span id="count" class="muted"></span>
@@ -265,6 +266,16 @@
<p class="muted small" data-i18n="hostHelp"></p> <p class="muted small" data-i18n="hostHelp"></p>
</section> </section>
<section class="set-section">
<h3 data-i18n="dmesgTitle">Kernel messages (dmesg)</h3>
<p id="dmesgStatus" class="docker-status"></p>
<label class="switch-row">
<input id="dmesgEnabled" type="checkbox" class="switch">
<span data-i18n="dmesgEnabled">Collect the kernel messages of this machine</span>
</label>
<p class="muted small" data-i18n="dmesgHelp"></p>
</section>
<section class="set-section span"> <section class="set-section span">
<h3 data-i18n="dockerTitle">Docker containers</h3> <h3 data-i18n="dockerTitle">Docker containers</h3>
<p id="dockerStatus" class="docker-status"></p> <p id="dockerStatus" class="docker-status"></p>