From 30018e09e273958e1346baa0a484b3d8671cc59b Mon Sep 17 00:00:00 2001 From: Cedric Date: Sat, 3 Oct 2026 10:05:38 +0200 Subject: [PATCH] Host system logs source (systemd journal or /var/log) New source, off by default and switched in Settings > Sources, that collects the system logs of the machine hosting the stack: - reads the systemd journal files directly (pure Go reader, no journalctl in the image), from /var/log/journal and /run/log/journal mounted read-only under /host; - falls back to following the text files of /var/log (syslog, messages, *.log) on hosts without journald; - positions saved in /data/hostlogs-state.json, HOST_LOGS_BACKFILL read when the source is turned on; - source_type "host", selectable in the Source filter; - compose mounts and group_add (HOST_LOGS_GID, adm by default), docs. Co-Authored-By: Claude Opus 5.5 (1M context) --- .env.example | 6 + README.fr.md | 30 +++ README.md | 30 +++ api.go | 23 +++ docker-compose.yml | 6 + hostlogs.go | 452 +++++++++++++++++++++++++++++++++++++++++++++ hostlogs_test.go | 206 +++++++++++++++++++++ journal.go | 174 +++++++++++++++++ main.go | 9 +- query.go | 10 +- web/app.js | 81 +++++++- web/index.html | 11 ++ 12 files changed, 1029 insertions(+), 9 deletions(-) create mode 100644 hostlogs.go create mode 100644 hostlogs_test.go create mode 100644 journal.go diff --git a/.env.example b/.env.example index a48e89b..299c193 100644 --- a/.env.example +++ b/.env.example @@ -19,3 +19,9 @@ EXPORT_MAX=100000 DOCKER_LOGS=on # History read from a container seen for the first time (e.g. 30m, 1h, 24h; 0 = only new lines) DOCKER_BACKFILL=1h +# Host system logs (enable them in Settings > Sources) +# Group allowed to read the host logs: 4 = adm on Debian/Ubuntu; or the systemd-journal group +# (getent group systemd-journal | cut -d: -f3) +HOST_LOGS_GID=4 +# History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new entries) +HOST_LOGS_BACKFILL=1h diff --git a/README.fr.md b/README.fr.md index 9e72ff8..4a05475 100644 --- a/README.fr.md +++ b/README.fr.md @@ -164,6 +164,32 @@ donc par [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy) passer que la liste des conteneurs, la lecture des logs, les événements et les informations du moteur (`GET` uniquement). +## Logs système de l'hôte + +Logstream peut aussi collecter les logs système de la machine qui héberge la stack, sans rien +configurer sur l'hôte. La source est **désactivée par défaut** : activez-la dans +**Paramètres > Sources > Logs système de l'hôte** (enregistré dans `/data/hostlogs.json`). + +- `docker-compose.yml` monte `/var/log` et `/run/log/journal` en lecture seule sous `/host`. +- Si l'hôte utilise systemd, Logstream lit directement les fichiers du **journal systemd** + (pas besoin de `journalctl` dans l'image) : **host** est le nom de la machine, **app** le + programme (`SYSLOG_IDENTIFIER`), la sévérité et la facility viennent du journal, et l'unité + systemd est conservée dans `unit`. Sinon, il suit les fichiers texte de `/var/log` (`syslog`, + `messages`, `*.log`), analysés comme des lignes syslog, avec le nom du fichier dans `log_file`. +- Ces logs ont la source `host` : le filtre **Source** permet de les afficher seuls. +- À l'activation, la dernière heure est lue d'abord (`HOST_LOGS_BACKFILL`) ; la position + atteinte est enregistrée dans `/data/hostlogs-state.json`, un redémarrage ne perd donc ni ne + duplique d'entrées. +- **Droits** : le conteneur tourne avec un utilisateur sans privilège et reçoit le groupe `adm` + (gid 4), qui peut lire le journal et `/var/log` sur Debian et Ubuntu. Sur d'autres systèmes, + réglez `HOST_LOGS_GID` dans `.env` sur le gid de `systemd-journal` + (`getent group systemd-journal | cut -d: -f3`). Paramètres > Sources affiche un message clair + si l'accès est refusé. +- Limites : les champs du journal compressés par journald (messages de plus de 512 octets, + compressés en zstd/lz4/xz) ne peuvent pas être décodés sans bibliothèque supplémentaire ; ils + sont comptés dans les Paramètres et affichés comme « (compressed journal entry) ». Les fichiers + texte tournés ou compressés (`*.1`, `*.gz`) ne sont pas lus. + ## Export CSV Le bouton **Exporter** (à côté du nombre de logs) télécharge tous les logs stockés qui @@ -247,6 +273,9 @@ résolutions. | `DOCKER_LOGS` | `on` dans compose | collecter les logs des conteneurs Docker locaux | | `DOCKER_HOST` | `tcp://docker-proxy:2375` dans compose | adresse de l'API Docker (`unix:///var/run/docker.sock` hors compose) | | `DOCKER_BACKFILL` | `1h` | historique lu pour un conteneur vu pour la première fois | +| `HOST_LOGS_GID` | `4` (adm) dans compose | groupe donné au conteneur pour lire les logs de l'hôte | +| `HOST_LOGS_BACKFILL` | `1h` | historique lu à l'activation de la source « logs système de l'hôte » | +| `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte | | `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) | | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion | @@ -303,6 +332,7 @@ Pour mettre à jour l'une d'elles : | `export.go` | export CSV en flux | | `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) | | `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources | +| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` | | `tags.go` | stockage des tags de couleur | | `api.go` | routes HTTP `/api/*` | | `web/` | interface (HTML, CSS, JavaScript simple, sans étape de build), embarquée dans le binaire ; les traductions sont dans `web/app.js` (`I18N`) | diff --git a/README.md b/README.md index 229fd47..05fafd7 100644 --- a/README.md +++ b/README.md @@ -147,6 +147,32 @@ colored and exported like syslog messages: therefore goes through [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy), which only lets through listing containers, reading logs, events and engine info (`GET` only). +## Host system logs + +Logstream can also collect the system logs of the machine hosting the stack, without +configuring anything on the host. The source is **off by default**: turn it on in +**Settings > Sources > Host system logs** (saved in `/data/hostlogs.json`). + +- `docker-compose.yml` mounts `/var/log` and `/run/log/journal` read-only under `/host`. +- When the host runs systemd, Logstream reads the **systemd journal** files directly (no + `journalctl` needed in the image): **host** is the machine name, **app** the program + (`SYSLOG_IDENTIFIER`), severity and facility come from the journal, and the systemd unit is + kept in `unit`. Otherwise it follows the text files of `/var/log` (`syslog`, `messages`, + `*.log`), parsed like syslog lines, with the file name in `log_file`. +- These logs have the `host` source: the **Source** filter shows them alone. +- When the source is turned on, the last hour is read first (`HOST_LOGS_BACKFILL`); the + position reached is saved in `/data/hostlogs-state.json`, so a restart neither loses nor + duplicates entries. +- **Permissions**: the container runs as an unprivileged user and gets the `adm` group + (gid 4), which can read the journal and `/var/log` on Debian and Ubuntu. On other systems, + set `HOST_LOGS_GID` in `.env` to the gid of `systemd-journal` + (`getent group systemd-journal | cut -d: -f3`). Settings > Sources shows a clear message when + access is denied. +- Limits: journal fields compressed by journald (messages longer than 512 bytes, compressed + with zstd/lz4/xz) cannot be decoded without extra libraries; they are counted in Settings and + shown as "(compressed journal entry)". Rotated or compressed text files (`*.1`, `*.gz`) are + not read. + ## CSV export The **Export** button (next to the log count) downloads every stored log matching the @@ -222,6 +248,9 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set | `DOCKER_LOGS` | `on` in compose | collect the logs of the local Docker containers | | `DOCKER_HOST` | `tcp://docker-proxy:2375` in compose | Docker API address (`unix:///var/run/docker.sock` outside compose) | | `DOCKER_BACKFILL` | `1h` | history read from a container seen for the first time | +| `HOST_LOGS_GID` | `4` (adm) in compose | group given to the container to read the host logs | +| `HOST_LOGS_BACKFILL` | `1h` | history read when the host system logs source is turned on | +| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted | | `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) | | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning | @@ -275,6 +304,7 @@ To update one of them: | `export.go` | streamed CSV export | | `docker.go` | Docker container logs (API, followers, positions, level detection) | | `syslogserver.go` | syslog listeners opened and closed from Settings > Sources | +| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower | | `tags.go` | color tag storage | | `api.go` | `/api/*` HTTP routes | | `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`) | diff --git a/api.go b/api.go index 1545cc9..60705d2 100644 --- a/api.go +++ b/api.go @@ -21,6 +21,7 @@ type API struct { exportMax int docker *DockerManager // nil when DOCKER_LOGS is off syslog *SyslogServer + host *HostLogs } func (a *API) Routes(mux *http.ServeMux) { @@ -42,6 +43,28 @@ func (a *API) Routes(mux *http.ServeMux) { mux.HandleFunc("PUT /api/syslog", a.syslogConfigure) mux.HandleFunc("GET /api/docker", a.dockerStatus) mux.HandleFunc("PUT /api/docker", a.dockerConfigure) + mux.HandleFunc("GET /api/hostlogs", a.hostLogsStatus) + mux.HandleFunc("PUT /api/hostlogs", a.hostLogsConfigure) +} + +// GET /api/hostlogs: state of the host system logs source (Settings > Sources). +func (a *API) hostLogsStatus(w http.ResponseWriter, r *http.Request) { + writeJSON(w, http.StatusOK, a.host.Status()) +} + +// PUT /api/hostlogs {"enabled": bool} +func (a *API) hostLogsConfigure(w http.ResponseWriter, r *http.Request) { + var cfg hostLogsConfig + if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&cfg); err != nil { + writeErr(w, http.StatusBadRequest, err) + return + } + if err := a.host.Configure(cfg); err != nil { + writeErr(w, http.StatusInternalServerError, err) + return + } + log.Printf("host system logs changed from %s: %+v", r.RemoteAddr, cfg) + writeJSON(w, http.StatusOK, a.host.Status()) } // GET /api/syslog: syslog reception state (Settings > Sources). diff --git a/docker-compose.yml b/docker-compose.yml index a31a447..b8f15ae 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -23,8 +23,14 @@ services: DOCKER_LOGS: ${DOCKER_LOGS:-on} # collecte des logs des conteneurs Docker DOCKER_HOST: tcp://docker-proxy:2375 # lecture seul de l'API Docker DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h} + HOST_LOGS_BACKFILL: ${HOST_LOGS_BACKFILL:-1h} # historique lu a l'activation des logs systeme de l'hote + group_add: + - "${HOST_LOGS_GID:-4}" # groupe autorise a lire les logs de l'hote (4 = adm sur Debian/Ubuntu) volumes: - logstream-data:/data # tags.json, docker.json (les choix des conteneurs) + # logs systeme de l'hote, en lecture seule (source a activer dans Reglages > Sources) + - /var/log:/host/var/log:ro # journal systemd persistant et fichiers texte + - /run/log/journal:/host/run/log/journal:ro # journal systemd volatile labels: logstream.exclude: "true" # pas de collect des logs logstream diff --git a/hostlogs.go b/hostlogs.go new file mode 100644 index 0000000..00d1493 --- /dev/null +++ b/hostlogs.go @@ -0,0 +1,452 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "io" + "io/fs" + "log" + "os" + "path/filepath" + "sort" + "strconv" + "strings" + "sync" + "syscall" + "time" + "unicode/utf8" +) + +// hostLogsConfig is saved in /data/hostlogs.json and edited in Settings > Sources. +type hostLogsConfig struct { + Enabled bool `json:"enabled"` +} + +// hostPos is where reading resumes in a journal file (by file ID) or a text +// file (by name, with its inode to detect rotations). Off -1: archived file +// read to the end. +type hostPos struct { + Off int64 `json:"off"` + Ino uint64 `json:"ino,omitempty"` +} + +// HostLogs collects the system logs of the machine hosting the stack: the +// systemd journal when there is one, else the text files of /var/log. The +// host directories are mounted read-only under root (/host by default). +type HostLogs struct { + root string + sink func(*Entry) + backfill time.Duration + cfgPath string + statePath string + wake chan struct{} + + // Owned by the Run goroutine. + pos map[string]hostPos + dirty bool + started bool // a first scan was done since enabling: new files are read from their start + + mu sync.Mutex + cfg hostLogsConfig + reset bool + mode string // "journal", "files" or "" (nothing found) + files int + read uint64 + compressed uint64 + errCode string + errDetail string +} + +func NewHostLogs(root, dataDir string, backfill time.Duration, sink func(*Entry)) *HostLogs { + h := &HostLogs{ + root: root, + sink: sink, + backfill: backfill, + cfgPath: filepath.Join(dataDir, "hostlogs.json"), + statePath: filepath.Join(dataDir, "hostlogs-state.json"), + wake: make(chan struct{}, 1), + pos: map[string]hostPos{}, + } + if b, err := os.ReadFile(h.cfgPath); err == nil { + _ = json.Unmarshal(b, &h.cfg) + } + if b, err := os.ReadFile(h.statePath); err == nil { + _ = json.Unmarshal(b, &h.pos) + h.started = len(h.pos) > 0 + } + return h +} + +// Run polls the host logs every second while the source is enabled. +func (h *HostLogs) Run(ctx context.Context) { + tick := time.NewTicker(time.Second) + defer tick.Stop() + n := 0 + for { + select { + case <-ctx.Done(): + h.saveState() + return + case <-tick.C: + case <-h.wake: + } + h.mu.Lock() + enabled, reset := h.cfg.Enabled, h.reset + h.reset = false + h.mu.Unlock() + if reset { + // Disabled then enabled again: start over from HOST_LOGS_BACKFILL ago + // rather than reading everything written in between. + h.pos, h.started, h.dirty = map[string]hostPos{}, false, true + } + if enabled { + h.scan(time.Now()) + } + if n++; n%5 == 0 || reset { + h.saveState() + } + } +} + +func (h *HostLogs) saveState() { + if !h.dirty { + return + } + h.dirty = false + if err := writeJSONFile(h.statePath, h.pos); err != nil { + log.Printf("host logs: saving positions: %v", err) + } +} + +func (h *HostLogs) setStatus(mode string, files int, code, detail string) { + h.mu.Lock() + h.mode, h.files, h.errCode, h.errDetail = mode, files, code, detail + h.mu.Unlock() +} + +// scan reads what was added since the previous poll. +func (h *HostLogs) scan(now time.Time) { + notBefore := time.Time{} + if !h.started { + notBefore = now.Add(-h.backfill) + } + defer func() { h.started = true }() + + var journals []string + for _, dir := range []string{"var/log/journal", "run/log/journal"} { + base := filepath.Join(h.root, dir) + for _, pat := range []string{"*.journal", "*/*.journal"} { + m, _ := filepath.Glob(filepath.Join(base, pat)) + journals = append(journals, m...) + } + } + if len(journals) > 0 { + h.scanJournals(journals, notBefore) + return + } + h.scanFiles(notBefore.IsZero()) +} + +func (h *HostLogs) scanJournals(paths []string, notBefore time.Time) { + seen := map[string]bool{} + var firstErr error + for _, p := range paths { + f, err := os.Open(p) + if err != nil { + firstErr = keepFirst(firstErr, err) + continue + } + hdr, err := readJournalHeader(f) + f.Close() + if err != nil { + continue // file being created, or not a journal + } + key := "j:" + hdr.fileID + seen[key] = true + pos, known := h.pos[key] + if known && pos.Off < 0 { + continue + } + from, nb := pos.Off, time.Time{} + if !known { + if hdr.archived && !notBefore.IsZero() && time.UnixMicro(int64(hdr.tailRealtimeUS)).Before(notBefore) { + h.pos[key], h.dirty = hostPos{Off: -1}, true // archived before the backfill window + continue + } + nb = notBefore + } + next, hdr, err := readJournal(p, from, nb, h.emitJournal) + if err != nil { + firstErr = keepFirst(firstErr, err) + continue + } + if hdr.archived { + next = -1 + } + if !known || next != pos.Off { + h.pos[key], h.dirty = hostPos{Off: next}, true + } + } + h.prune("j:", seen) + code, detail := "", "" + if firstErr != nil && len(seen) == 0 { + code, detail = errCode(firstErr), firstErr.Error() + } + h.setStatus("journal", len(seen), code, detail) +} + +func keepFirst(first, err error) error { + if first != nil { + return first + } + return err +} + +func errCode(err error) string { + if errors.Is(err, fs.ErrPermission) { + return "permission" + } + return "read" +} + +func (h *HostLogs) prune(prefix string, seen map[string]bool) { + for k := range h.pos { + if strings.HasPrefix(k, prefix) && !seen[k] { + delete(h.pos, k) + h.dirty = true + } + } +} + +// hostLogFile reports the classic text logs of /var/log (rotated and +// compressed copies are left out). +func hostLogFile(name string) bool { + return name == "syslog" || name == "messages" || strings.HasSuffix(name, ".log") +} + +const maxHostRead = 4 << 20 // per file and per poll + +// scanFiles follows the text files of /var/log, for hosts without journald. +// Files present at the first scan are read from their end (only new lines). +func (h *HostLogs) scanFiles(fromStart bool) { + dir := filepath.Join(h.root, "var/log") + ents, err := os.ReadDir(dir) + if err != nil { + code := errCode(err) + if errors.Is(err, fs.ErrNotExist) { + code = "not_mounted" + } + h.setStatus("", 0, code, err.Error()) + return + } + seen := map[string]bool{} + var firstErr error + for _, de := range ents { + if !de.Type().IsRegular() || !hostLogFile(de.Name()) { + continue + } + name := de.Name() + key := "f:" + name + fi, err := de.Info() + if err != nil { + continue + } + var ino uint64 + if st, ok := fi.Sys().(*syscall.Stat_t); ok { + ino = uint64(st.Ino) + } + pos, known := h.pos[key] + switch { + case !known && !fromStart: + pos = hostPos{Off: fi.Size(), Ino: ino} + case !known || pos.Ino != ino || fi.Size() < pos.Off: + pos = hostPos{Off: 0, Ino: ino} // new, rotated or truncated file + } + if fi.Size() > pos.Off { + off, err := h.readFile(filepath.Join(dir, name), name, pos.Off) + if err != nil { + firstErr = keepFirst(firstErr, err) + continue // not readable: not counted as followed + } + pos.Off = off + } + seen[key] = true + if old, ok := h.pos[key]; !ok || old != pos { + h.pos[key], h.dirty = pos, true + } + } + h.prune("f:", seen) + code, detail := "", "" + if firstErr != nil && len(seen) == 0 { + code, detail = errCode(firstErr), firstErr.Error() + } + mode := "files" + if len(seen) == 0 && code == "" { + mode, code = "", "empty" + } + h.setStatus(mode, len(seen), code, detail) +} + +// readFile sends the complete lines written after off and returns the new offset. +func (h *HostLogs) readFile(path, name string, off int64) (int64, error) { + f, err := os.Open(path) + if err != nil { + return off, err + } + defer f.Close() + buf, err := io.ReadAll(io.NewSectionReader(f, off, maxHostRead)) + if err != nil { + return off, err + } + end := bytes.LastIndexByte(buf, '\n') + if end < 0 { + if len(buf) < maxHostRead { + return off, nil // partial line: wait for its end + } + end = len(buf) - 1 // line longer than the read window: cut it + } + now := time.Now() + fac := fileFacility(name) + for _, line := range bytes.Split(buf[:end+1], []byte{'\n'}) { + if len(bytes.TrimSpace(line)) == 0 { + continue + } + e := ParseSyslog(line, "", "file", now) + if e.Host == "" { + e.Host = "localhost" + } + if n, ok := detectLevel(e.Message); ok { + e.SevNum, e.Severity = n, severityNames[n] + } else { + e.SevNum, e.Severity = 6, "info" + } + if fac >= 0 { + e.Facility = facilityNames[fac] + } + e.SourceType = "host" + e.Extra = map[string]string{"log_file": "/var/log/" + name} + h.sink(e) + h.count(1, 0) + } + return off + int64(end) + 1, nil +} + +// fileFacility guesses the facility from the usual Debian/RHEL file names. +func fileFacility(name string) int { + switch strings.TrimSuffix(name, ".log") { + case "kern": + return 0 + case "mail", "maillog": + return 2 + case "daemon": + return 3 + case "auth", "secure": + return 4 + case "cron": + return 9 + } + return -1 +} + +func (h *HostLogs) count(read, compressed uint64) { + h.mu.Lock() + h.read += read + h.compressed += compressed + h.mu.Unlock() +} + +// emitJournal turns a journal entry into an Entry. +func (h *HostLogs) emitJournal(je *journalEntry) { + f := je.Fields + msg := f["MESSAGE"] + if msg == "" && je.Compressed > 0 { + msg = "(compressed journal entry: read it with journalctl)" + } + h.count(1, uint64(je.Compressed)) + if strings.TrimSpace(msg) == "" { + return + } + if !utf8.ValidString(msg) { + msg = strings.ToValidUTF8(msg, "�") + } + sev := 6 + if n, err := strconv.Atoi(f["PRIORITY"]); err == nil && n >= 0 && n <= 7 { + sev = n + } + fac := 1 // user + switch { + case f["_TRANSPORT"] == "kernel": + fac = 0 + case f["_SYSTEMD_UNIT"] != "": + fac = 3 // daemon + } + if n, err := strconv.Atoi(f["SYSLOG_FACILITY"]); err == nil && n >= 0 && n < len(facilityNames) { + fac = n + } + app := firstNonEmpty(f["SYSLOG_IDENTIFIER"], f["_COMM"], strings.TrimSuffix(f["_SYSTEMD_UNIT"], ".service")) + host := firstNonEmpty(f["_HOSTNAME"], "localhost") + h.sink(&Entry{ + Time: je.Realtime, + Received: time.Now(), + Host: host, + App: app, + ProcID: firstNonEmpty(f["SYSLOG_PID"], f["_PID"]), + Facility: facilityNames[fac], + Severity: severityNames[sev], + SevNum: sev, + Message: strings.TrimRight(msg, "\n"), + Proto: "journal", + SourceType: "host", + Extra: map[string]string{"unit": f["_SYSTEMD_UNIT"]}, + }) +} + +func firstNonEmpty(vals ...string) string { + for _, v := range vals { + if v != "" { + return v + } + } + return "" +} + +// Configure saves and applies the configuration. +func (h *HostLogs) Configure(cfg hostLogsConfig) error { + h.mu.Lock() + if h.cfg.Enabled && !cfg.Enabled { + h.reset = true + h.mode, h.files, h.errCode, h.errDetail = "", 0, "", "" + } + h.cfg = cfg + h.mu.Unlock() + select { + case h.wake <- struct{}{}: + default: + } + return writeJSONFile(h.cfgPath, cfg) +} + +// Status is the state shown in Settings > Sources. +func (h *HostLogs) Status() map[string]any { + h.mu.Lock() + defer h.mu.Unlock() + dirs := []string{} + for _, d := range []string{"var/log/journal", "run/log/journal", "var/log"} { + if _, err := os.Stat(filepath.Join(h.root, d)); err == nil { + dirs = append(dirs, "/"+d) + } + } + sort.Strings(dirs) + return map[string]any{ + "enabled": h.cfg.Enabled, + "mode": h.mode, + "files": h.files, + "read": h.read, + "compressed": h.compressed, + "code": h.errCode, + "error": h.errDetail, + "mounted": dirs, + } +} diff --git a/hostlogs_test.go b/hostlogs_test.go new file mode 100644 index 0000000..05b9a7a --- /dev/null +++ b/hostlogs_test.go @@ -0,0 +1,206 @@ +package main + +import ( + "encoding/binary" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +// fakeJournal builds a minimal journal file: header, data objects, entries. +type fakeJournal struct { + compact bool + buf []byte + seq uint64 + tail uint64 +} + +func newFakeJournal(compact bool) *fakeJournal { + j := &fakeJournal{compact: compact, buf: make([]byte, 272)} + copy(j.buf, jSignature) + if compact { + binary.LittleEndian.PutUint32(j.buf[12:], jIncompatCompact) + } + j.buf[16] = 1 // online + copy(j.buf[24:40], "0123456789abcdef") + binary.LittleEndian.PutUint64(j.buf[88:], 272) + return j +} + +func (j *fakeJournal) object(typ, flags byte, body []byte) uint64 { + for len(j.buf)%8 != 0 { + j.buf = append(j.buf, 0) + } + off := uint64(len(j.buf)) + h := make([]byte, jObjHeaderSize) + h[0], h[1] = typ, flags + binary.LittleEndian.PutUint64(h[8:], uint64(jObjHeaderSize+len(body))) + j.buf = append(append(j.buf, h...), body...) + j.tail = off + binary.LittleEndian.PutUint64(j.buf[136:], off) + return off +} + +func (j *fakeJournal) data(field string, flags byte) uint64 { + n := 48 + if j.compact { + n = 56 + } + return j.object(jObjData, flags, append(make([]byte, n), field...)) +} + +// entry appends an entry; linked=false leaves it unfinished (tail seqnum not updated). +func (j *fakeJournal) entry(t time.Time, linked bool, fields ...string) { + var items []byte + for _, f := range fields { + flags := byte(0) + if strings.HasPrefix(f, "!") { // compressed data object + f, flags = f[1:], 4 + } + off := j.data(f, flags) + if j.compact { + items = binary.LittleEndian.AppendUint32(items, uint32(off)) + } else { + items = binary.LittleEndian.AppendUint64(items, off) + items = binary.LittleEndian.AppendUint64(items, 0) + } + } + j.seq++ + body := make([]byte, 48) + binary.LittleEndian.PutUint64(body[0:], j.seq) + binary.LittleEndian.PutUint64(body[8:], uint64(t.UnixMicro())) + j.object(jObjEntry, 0, append(body, items...)) + if linked { + binary.LittleEndian.PutUint64(j.buf[160:], j.seq) + binary.LittleEndian.PutUint64(j.buf[192:], uint64(t.UnixMicro())) + } +} + +func (j *fakeJournal) write(t *testing.T, path string) { + t.Helper() + if err := os.WriteFile(path, j.buf, 0o644); err != nil { + t.Fatal(err) + } +} + +func TestReadJournal(t *testing.T) { + for _, compact := range []bool{false, true} { + path := filepath.Join(t.TempDir(), "system.journal") + now := time.Now() + j := newFakeJournal(compact) + j.entry(now.Add(-2*time.Hour), true, "MESSAGE=too old", "PRIORITY=6") + j.entry(now.Add(-time.Minute), true, "MESSAGE=Started cron.", "PRIORITY=5", "SYSLOG_IDENTIFIER=systemd", "_HOSTNAME=srv1", "_PID=1") + j.write(t, path) + + var got []*journalEntry + emit := func(e *journalEntry) { got = append(got, e) } + next, _, err := readJournal(path, 0, now.Add(-time.Hour), emit) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Fields["MESSAGE"] != "Started cron." || got[0].Fields["_HOSTNAME"] != "srv1" { + t.Fatalf("compact=%v: got %+v", compact, got) + } + + // A new complete entry and one still being written. + j.entry(now, true, "MESSAGE=second", "!MESSAGE=big") + j.entry(now, false, "MESSAGE=unfinished") + j.write(t, path) + got = nil + next2, _, err := readJournal(path, next, time.Time{}, emit) + if err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Fields["MESSAGE"] != "second" || got[0].Compressed != 1 { + t.Fatalf("compact=%v: resumed read got %d entries", compact, len(got)) + } + + // Once linked, the unfinished entry is read from where reading stopped. + binary.LittleEndian.PutUint64(j.buf[160:], j.seq) + j.write(t, path) + got = nil + if _, _, err := readJournal(path, next2, time.Time{}, emit); err != nil { + t.Fatal(err) + } + if len(got) != 1 || got[0].Fields["MESSAGE"] != "unfinished" { + t.Fatalf("compact=%v: unfinished entry got %+v", compact, got) + } + } +} + +func TestHostLogsJournal(t *testing.T) { + root := t.TempDir() + dir := filepath.Join(root, "var/log/journal/machine") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + j := newFakeJournal(true) + j.entry(time.Now(), true, "MESSAGE=Accepted publickey", "PRIORITY=6", "SYSLOG_FACILITY=10", "SYSLOG_IDENTIFIER=sshd", "_PID=42", "_HOSTNAME=srv1", "_SYSTEMD_UNIT=ssh.service") + j.entry(time.Now(), true, "MESSAGE=oops", "PRIORITY=3", "_TRANSPORT=kernel", "_HOSTNAME=srv1") + j.write(t, filepath.Join(dir, "system.journal")) + + var got []*Entry + h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) }) + h.scan(time.Now()) + if len(got) != 2 { + t.Fatalf("got %d entries", len(got)) + } + e := got[0] + if e.App != "sshd" || e.ProcID != "42" || e.Facility != "authpriv" || e.Severity != "info" || e.Host != "srv1" || e.SourceType != "host" || e.Extra["unit"] != "ssh.service" { + t.Fatalf("entry %+v", e) + } + if got[1].Facility != "kern" || got[1].Severity != "err" { + t.Fatalf("kernel entry %+v", got[1]) + } + h.scan(time.Now()) // nothing new + if len(got) != 2 { + t.Fatalf("re-read: %d entries", len(got)) + } + if st := h.Status(); st["mode"] != "journal" || st["files"] != 1 { + t.Fatalf("status %+v", st) + } +} + +func TestHostLogsFiles(t *testing.T) { + root := t.TempDir() + dir := filepath.Join(root, "var/log") + if err := os.MkdirAll(dir, 0o755); err != nil { + t.Fatal(err) + } + auth := filepath.Join(dir, "auth.log") + if err := os.WriteFile(auth, []byte("Oct 3 07:00:00 srv1 sshd[1]: old line\n"), 0o644); err != nil { + t.Fatal(err) + } + _ = os.WriteFile(filepath.Join(dir, "auth.log.1"), []byte("rotated\n"), 0o644) + + var got []*Entry + h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) }) + h.scan(time.Now()) // existing content is skipped + if len(got) != 0 { + t.Fatalf("first scan read %d lines", len(got)) + } + f, _ := os.OpenFile(auth, os.O_APPEND|os.O_WRONLY, 0) + _, _ = f.WriteString("Oct 3 08:00:00 srv1 sshd[7]: Failed password for root\nOct 3 08:00:01 srv1 sshd[7]: partial") + f.Close() + h.scan(time.Now()) + if len(got) != 1 { + t.Fatalf("got %d lines", len(got)) + } + e := got[0] + if e.Host != "srv1" || e.App != "sshd" || e.ProcID != "7" || e.Facility != "auth" || e.Extra["log_file"] != "/var/log/auth.log" || e.Proto != "file" { + t.Fatalf("entry %+v", e) + } + if st := h.Status(); st["mode"] != "files" || st["files"] != 1 { + t.Fatalf("status %+v", st) + } +} + +func TestHostLogsNotMounted(t *testing.T) { + h := NewHostLogs(filepath.Join(t.TempDir(), "none"), t.TempDir(), time.Hour, func(*Entry) {}) + h.scan(time.Now()) + if st := h.Status(); st["code"] != "not_mounted" { + t.Fatalf("status %+v", st) + } +} diff --git a/journal.go b/journal.go new file mode 100644 index 0000000..7f242fb --- /dev/null +++ b/journal.go @@ -0,0 +1,174 @@ +package main + +import ( + "bufio" + "bytes" + "encoding/binary" + "encoding/hex" + "errors" + "io" + "os" + "time" +) + +// Minimal reader of systemd journal files (*.journal), enough to follow them +// without journalctl: objects are walked in file order, which is the order +// entries were written. Format: https://systemd.io/JOURNAL_FILE_FORMAT/ + +const ( + jHeaderMin = 208 // header fields used below end at offset 208 + jObjHeaderSize = 16 + + jObjData = 1 + jObjEntry = 3 + + jIncompatCompact = 1 << 4 + jStateArchived = 2 + jObjCompressed = 1<<0 | 1<<1 | 1<<2 // XZ, LZ4, ZSTD: not decoded (no stdlib codec) +) + +var jSignature = []byte("LPKSHHRH") + +// jHeader holds the header fields the reader needs. +type jHeader struct { + compact bool + archived bool + fileID string + headerSize uint64 + tailObject uint64 // offset of the last object + tailSeqnum uint64 // seqnum of the last complete entry + tailRealtimeUS uint64 // realtime of the last entry (µs since the epoch) +} + +func readJournalHeader(f io.ReaderAt) (jHeader, error) { + b := make([]byte, jHeaderMin) + if _, err := f.ReadAt(b, 0); err != nil { + return jHeader{}, err + } + if !bytes.Equal(b[:8], jSignature) { + return jHeader{}, errors.New("not a journal file") + } + le := binary.LittleEndian + h := jHeader{ + compact: le.Uint32(b[12:])&jIncompatCompact != 0, + archived: b[16] == jStateArchived, + fileID: hex.EncodeToString(b[24:40]), + headerSize: le.Uint64(b[88:]), + tailObject: le.Uint64(b[136:]), + tailSeqnum: le.Uint64(b[160:]), + tailRealtimeUS: le.Uint64(b[192:]), + } + if h.headerSize < jHeaderMin { + return h, errors.New("journal header too small") + } + return h, nil +} + +// journalEntry is one entry: its time and its "FIELD=value" pairs. +type journalEntry struct { + Realtime time.Time + Fields map[string]string + Compressed int // fields that could not be read (compressed data) +} + +// readJournal reads the entries complete after offset `from` (0 = start of +// the file) and returns the offset to resume from. Entries older than +// `notBefore` are skipped without decoding their fields. +func readJournal(path string, from int64, notBefore time.Time, emit func(*journalEntry)) (next int64, h jHeader, err error) { + f, err := os.Open(path) + if err != nil { + return from, h, err + } + defer f.Close() + if h, err = readJournalHeader(f); err != nil { + return from, h, err + } + if from < int64(h.headerSize) { + from = int64(h.headerSize) + } + end := int64(h.tailObject) + r := bufio.NewReaderSize(io.NewSectionReader(f, from, 1<<62), 64*1024) + le := binary.LittleEndian + hdr := make([]byte, jObjHeaderSize) + off := from + for off <= end { + if _, err := io.ReadFull(r, hdr); err != nil { + return off, h, nil // object still being written + } + typ, size := hdr[0], int64(le.Uint64(hdr[8:])) + if size < jObjHeaderSize { + return off, h, nil + } + body := size - jObjHeaderSize + if typ == jObjEntry && body >= 48 && body < 1<<20 { + obj := make([]byte, body) + if _, err := io.ReadFull(r, obj); err != nil { + return off, h, nil + } + seq := le.Uint64(obj[0:]) + if seq == 0 || seq > h.tailSeqnum { + return off, h, nil // not linked yet: retry at the next poll + } + rt := time.UnixMicro(int64(le.Uint64(obj[8:]))) + if !rt.Before(notBefore) { + emit(readEntryFields(f, h.compact, rt, obj[48:])) + } + } else if _, err := r.Discard(int(body)); err != nil { + return off, h, nil + } + next := (off + size + 7) &^ 7 // objects are 8-byte aligned + if pad := next - off - size; pad > 0 { + if _, err := r.Discard(int(pad)); err != nil { + return next, h, nil // the object is complete: resume after it + } + } + off = next + } + return off, h, nil +} + +// readEntryFields resolves the data objects referenced by an entry. +func readEntryFields(f io.ReaderAt, compact bool, rt time.Time, items []byte) *journalEntry { + le := binary.LittleEndian + e := &journalEntry{Realtime: rt, Fields: make(map[string]string, 16)} + step := 16 + if compact { + step = 4 + } + payloadAt := int64(64) + if compact { + payloadAt = 72 + } + hdr := make([]byte, jObjHeaderSize) + for i := 0; i+step <= len(items); i += step { + var off int64 + if compact { + off = int64(le.Uint32(items[i:])) + } else { + off = int64(le.Uint64(items[i:])) + } + if off == 0 { + continue + } + if _, err := f.ReadAt(hdr, off); err != nil || hdr[0] != jObjData { + continue + } + if hdr[1]&jObjCompressed != 0 { + e.Compressed++ + continue + } + size := int64(le.Uint64(hdr[8:])) + n := size - payloadAt + if n <= 0 || n > 1<<20 { + continue + } + p := make([]byte, n) + if _, err := f.ReadAt(p, off+payloadAt); err != nil { + continue + } + if k := bytes.IndexByte(p, '='); k > 0 { + e.Fields[string(p[:k])] = string(p[k+1:]) + } + } + return e +} diff --git a/main.go b/main.go index 8b938a6..9849d22 100644 --- a/main.go +++ b/main.go @@ -1,4 +1,4 @@ -// Logstream: a syslog (UDP/TCP) sink stored in VictoriaLogs, +// Logstream: a syslog (UDP/TCP), Docker and host system logs sink stored in VictoriaLogs, // with a web interface to browse and search the logs. package main @@ -38,6 +38,8 @@ type config struct { dockerLogs bool dockerHost string backfill time.Duration + hostRoot string + hostFill time.Duration batchSize int queueSize int flushEvery time.Duration @@ -89,6 +91,8 @@ func main() { dockerLogs: getenvBool("DOCKER_LOGS", false), dockerHost: getenv("DOCKER_HOST", "unix:///var/run/docker.sock"), backfill: getenvDuration("DOCKER_BACKFILL", time.Hour), + hostRoot: getenv("HOST_LOGS_ROOT", "/host"), + hostFill: getenvDuration("HOST_LOGS_BACKFILL", time.Hour), batchSize: getenvInt("BATCH_SIZE", 1000), queueSize: getenvInt("QUEUE_SIZE", 100000), flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond, @@ -139,6 +143,9 @@ func main() { log.Printf("docker logs enabled through %s", cfg.dockerHost) } } + // System logs of the host (journal or /var/log), off until enabled in Settings > Sources. + api.host = NewHostLogs(cfg.hostRoot, cfg.dataDir, cfg.hostFill, sink) + go api.host.Run(ctx) api.Routes(mux) mux.Handle("GET /", http.FileServer(http.FS(static))) diff --git a/query.go b/query.go index 8f09a73..457d65f 100644 --- a/query.go +++ b/query.go @@ -17,7 +17,7 @@ type Filter struct { Host string App string Severity int // highest severity number included (0 = emerg … 7 = debug), -1 = all - Source string // "syslog", "docker" or "" for all + Source string // "syslog", "docker", "host" or "" for all } var rangeDurations = map[string]time.Duration{ @@ -119,10 +119,10 @@ func (f Filter) filterExpr() string { parts = append(parts, "app:="+strconv.Quote(f.App)) } switch f.Source { - case "docker": - parts = append(parts, `source_type:="docker"`) + case "docker", "host": + parts = append(parts, `source_type:=`+strconv.Quote(f.Source)) case "syslog": // also matches logs stored before source_type existed - parts = append(parts, `!(source_type:="docker")`) + parts = append(parts, `!(source_type:in("docker","host"))`) } if f.Severity >= 0 && f.Severity < 7 { names := make([]string, 0, 8) @@ -209,7 +209,7 @@ func (m *Matcher) Match(e *Entry) bool { if m.f.App != "" && e.App != m.f.App { return false } - if (m.f.Source == "docker") != (e.SourceType == "docker") && m.f.Source != "" { + if m.f.Source != "" && m.f.Source != e.SourceType { return false } if m.f.Severity >= 0 && e.SevNum > m.f.Severity { diff --git a/web/app.js b/web/app.js index e9a77ec..21bb7bc 100644 --- a/web/app.js +++ b/web/app.js @@ -67,6 +67,19 @@ const I18N = { fHostName: 'host name (DNS)', fHostIp: 'host IP', clickHost: 'Click to filter on this host', clickApp: 'Click to filter on this app', sourceAria: 'Source', srcAll: 'All sources', tabSources: 'Sources', + srcHost: 'Host system', + hostTitle: 'Host system logs', + hostEnabled: 'Collect the system logs of this machine', + hostOff: 'Off: the system logs of the machine hosting Logstream are not collected.', + hostWaiting: 'Starting…', + hostJournal: ({ n, r }) => `Reading the systemd journal (${n} files): ${r} entries since startup.`, + hostFiles: ({ n, r }) => `Following ${n} files of /var/log: ${r} lines since startup.`, + hostCompressed: (n) => ` ${n} compressed fields skipped (long messages, readable with journalctl).`, + host_not_mounted: 'No host log directory found: mount /var/log and /run/log/journal under /host (see docker-compose.yml).', + host_permission: 'Permission denied: give the container a group allowed to read the logs (HOST_LOGS_GID in .env, see the README). ', + host_empty: 'No systemd journal and no log file found in /var/log.', + host_read: 'Cannot read the host logs: ', + hostHelp: 'Reads the systemd journal of the host (/var/log/journal, /run/log/journal), or the text files of /var/log (syslog, messages, *.log) when there is no journal. Directories are mounted read-only. When turned on, the last hour is read first (HOST_LOGS_BACKFILL).', syslogEnabled: 'Receive syslog messages', syslogProtocols: 'Protocols', syslogPort: 'Port', syslogListening: ({ port, protos }) => `Listening on port ${port} (${protos}).`, syslogOff: 'Syslog reception is off: syslog messages are ignored (Docker logs are still collected).', @@ -88,6 +101,7 @@ const I18N = { dockerHelp: 'Logs are read through docker-socket-proxy, a read-only gateway: Logstream can list containers and read their logs, nothing else. Choices apply per compose service (or container name), so they survive container re-creations.', fContainer: 'container', fContainerId: 'container ID', fImage: 'image', fProject: 'compose project', fService: 'compose service', fStream: 'stream', fSourceType: 'source', + fUnit: 'systemd unit', fLogFile: 'log file', export: 'Export', exportCsvHint: 'Comma separated, UTF-8', exportExcel: 'CSV for Excel', exportExcelHint: 'Semicolon separated, for Excel in French', exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`, @@ -196,6 +210,19 @@ const I18N = { fHostName: 'nom d\'hôte (DNS)', fHostIp: 'IP de l\'hôte', clickHost: 'Cliquer pour filtrer sur cet hôte', clickApp: 'Cliquer pour filtrer sur cette appli', sourceAria: 'Source', srcAll: 'Toutes les sources', tabSources: 'Sources', + srcHost: 'Système hôte', + hostTitle: 'Logs système de l\'hôte', + hostEnabled: 'Collecter les logs système de cette machine', + hostOff: 'Désactivé : les logs système de la machine qui héberge Logstream ne sont pas collectés.', + hostWaiting: 'Démarrage…', + hostJournal: ({ n, r }) => `Lecture du journal systemd (${n} fichiers) : ${r} entrées depuis le démarrage.`, + hostFiles: ({ n, r }) => `Suivi de ${n} fichiers de /var/log : ${r} lignes depuis le démarrage.`, + hostCompressed: (n) => ` ${n} champs compressés ignorés (messages longs, lisibles avec journalctl).`, + host_not_mounted: 'Aucun répertoire de logs de l\'hôte trouvé : montez /var/log et /run/log/journal sous /host (voir docker-compose.yml).', + host_permission: 'Accès refusé : donnez au conteneur un groupe autorisé à lire les logs (HOST_LOGS_GID dans .env, voir le README). ', + host_empty: 'Ni journal systemd ni fichier de log trouvé dans /var/log.', + host_read: 'Lecture des logs de l\'hôte impossible : ', + hostHelp: 'Lit le journal systemd de l\'hôte (/var/log/journal, /run/log/journal), ou les fichiers texte de /var/log (syslog, messages, *.log) s\'il n\'y a pas de journal. Les répertoires sont montés en lecture seule. À l\'activation, la dernière heure est lue d\'abord (HOST_LOGS_BACKFILL).', syslogEnabled: 'Recevoir les messages syslog', syslogProtocols: 'Protocoles', syslogPort: 'Port', syslogListening: ({ port, protos }) => `Écoute sur le port ${port} (${protos}).`, syslogOff: 'Réception syslog désactivée : les messages syslog sont ignorés (les logs Docker sont toujours collectés).', @@ -217,6 +244,7 @@ const I18N = { dockerHelp: 'Les logs sont lus via docker-socket-proxy, une passerelle en lecture seule : Logstream peut lister les conteneurs et lire leurs logs, rien d\'autre. Les choix s\'appliquent par service compose (ou nom de conteneur), ils survivent donc à la recréation des conteneurs.', fContainer: 'conteneur', fContainerId: 'ID du conteneur', fImage: 'image', fProject: 'projet compose', fService: 'service compose', fStream: 'flux', fSourceType: 'source', + fUnit: 'unité systemd', fLogFile: 'fichier de log', export: 'Exporter', exportCsvHint: 'Séparateur virgule, UTF-8', exportExcel: 'CSV pour Excel', exportExcelHint: 'Séparateur point-virgule, pour Excel en français', exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`, @@ -447,6 +475,7 @@ function setLang(next) { renderPurge(); renderInterface(); renderSyslog(); + renderHost(); renderDocker(); } @@ -805,14 +834,14 @@ function updateCount(tookMs) { } function detailsHTML(r) { - const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg']; + const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'unit', 'log_file', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg']; const hidden = new Set(['_stream_id', '_stream', 'sevnum']); if (r.msg_time) hidden.add('_time'); // same as the reception time const keys = order.filter((k) => r[k] != null && r[k] !== '' && !hidden.has(k)) .concat(Object.keys(r).filter((k) => !order.includes(k) && !hidden.has(k)).sort()); const label = { container: t('fContainer'), container_id: t('fContainerId'), image: t('fImage'), compose_project: t('fProject'), - compose_service: t('fService'), stream: t('fStream'), source_type: t('fSourceType'), + compose_service: t('fService'), stream: t('fStream'), source_type: t('fSourceType'), unit: t('fUnit'), log_file: t('fLogFile'), msg_time: t('fTime'), host_name: t('fHostName'), host_ip: t('fHostIp'), received: t('fReceived'), _time: t('fTime'), _msg: t('fMsg'), procid: t('fPid'), sd: t('fSd') }; const val = (k) => (k === '_time' || k === 'received' || k === 'msg_time' ? `${esc(fmtFull(r[k]))} (${esc(r[k])})` @@ -1726,6 +1755,48 @@ $('#syslogProtos').addEventListener('click', (ev) => { configureSyslog({ [b.dataset.proto]: !syslogState[b.dataset.proto] }); }); +/* ================= Host system logs ================= */ + +let hostState = null; + +async function loadHost() { + try { hostState = await api('/api/hostlogs'); } catch { hostState = null; } + renderHost(); +} + +function renderHost() { + const s = hostState; + const status = $('#hostStatus'); + if (!s) { status.textContent = ''; return; } + $('#hostEnabled').checked = s.enabled; + let text = t('hostWaiting'); + let cls = 'muted'; + if (!s.enabled) { + text = t('hostOff'); + } else if (s.code) { + text = t('host_' + s.code) + (s.code === 'read' || s.code === 'permission' ? s.error || '' : ''); + cls = 'bad'; + } else if (s.mode === 'journal') { + text = t('hostJournal', { n: s.files, r: fmtNum(s.read) }) + (s.compressed ? t('hostCompressed', fmtNum(s.compressed)) : ''); + cls = 'good'; + } else if (s.mode === 'files') { + text = t('hostFiles', { n: s.files, r: fmtNum(s.read) }); + cls = 'good'; + } + status.textContent = text; + status.className = 'docker-status ' + cls; +} + +async function configureHost(enabled) { + hostState = { ...(hostState || {}), enabled }; + renderHost(); + try { hostState = await api('/api/hostlogs', { method: 'PUT', body: { enabled } }); } catch (e) { toast(e.message); } + renderHost(); + setTimeout(loadHost, 1500); // the first scan runs in the background +} + +$('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked)); + /* ================= Docker source ================= */ let dockerState = null; @@ -1839,7 +1910,10 @@ for (const [id, on] of [['#dockerAll', true], ['#dockerNone', false]]) { } // Keep the list fresh while the Sources tab is open. setInterval(() => { - if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) loadDocker(); + if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) { + loadDocker(); + loadHost(); + } }, 4000); /* ================= Purge ================= */ @@ -2002,6 +2076,7 @@ $('#settingsBtn').addEventListener('click', () => { renderInterface(); loadPurgeStatus(); loadSyslog(); + loadHost(); loadDocker(); showSettingsTab(store.get('settingsTab', 'locale')); $('#settingsDlg').showModal(); diff --git a/web/index.html b/web/index.html index a8ea7f7..17dd07e 100644 --- a/web/index.html +++ b/web/index.html @@ -70,6 +70,7 @@ + @@ -209,6 +210,16 @@ +
+

Host system logs

+

+ +

+
+

Docker containers