Merge pull request 'Source « logs système de l'hôte » (journal systemd ou /var/log)' (#6) from feat/logs-systeme into main

.env.example

@@ -19,3 +19,9 @@ EXPORT_MAX=100000
 DOCKER_LOGS=on
 # History read from a container seen for the first time (e.g. 30m, 1h, 24h; 0 = only new lines)
 DOCKER_BACKFILL=1h
+# Host system logs (enable them in Settings > Sources)
+# Group allowed to read the host logs: 4 = adm on Debian/Ubuntu; or the systemd-journal group
+# (getent group systemd-journal | cut -d: -f3)
+HOST_LOGS_GID=4
+# History read when the source is turned on (e.g. 30m, 1h, 24h; 0 = only new entries)
+HOST_LOGS_BACKFILL=1h

README.fr.md

@@ -164,6 +164,32 @@ donc par [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy)
 passer que la liste des conteneurs, la lecture des logs, les événements et les informations du
 moteur (`GET` uniquement).
 
+## Logs système de l'hôte
+
+Logstream peut aussi collecter les logs système de la machine qui héberge la stack, sans rien
+configurer sur l'hôte. La source est **désactivée par défaut** : activez-la dans
+**Paramètres > Sources > Logs système de l'hôte** (enregistré dans `/data/hostlogs.json`).
+
+- `docker-compose.yml` monte `/var/log` et `/run/log/journal` en lecture seule sous `/host`.
+- Si l'hôte utilise systemd, Logstream lit directement les fichiers du **journal systemd**
+  (pas besoin de `journalctl` dans l'image) : **host** est le nom de la machine, **app** le
+  programme (`SYSLOG_IDENTIFIER`), la sévérité et la facility viennent du journal, et l'unité
+  systemd est conservée dans `unit`. Sinon, il suit les fichiers texte de `/var/log` (`syslog`,
+  `messages`, `*.log`), analysés comme des lignes syslog, avec le nom du fichier dans `log_file`.
+- Ces logs ont la source `host` : le filtre **Source** permet de les afficher seuls.
+- À l'activation, la dernière heure est lue d'abord (`HOST_LOGS_BACKFILL`) ; la position
+  atteinte est enregistrée dans `/data/hostlogs-state.json`, un redémarrage ne perd donc ni ne
+  duplique d'entrées.
+- **Droits** : le conteneur tourne avec un utilisateur sans privilège et reçoit le groupe `adm`
+  (gid 4), qui peut lire le journal et `/var/log` sur Debian et Ubuntu. Sur d'autres systèmes,
+  réglez `HOST_LOGS_GID` dans `.env` sur le gid de `systemd-journal`
+  (`getent group systemd-journal | cut -d: -f3`). Paramètres > Sources affiche un message clair
+  si l'accès est refusé.
+- Limites : les champs du journal compressés par journald (messages de plus de 512 octets,
+  compressés en zstd/lz4/xz) ne peuvent pas être décodés sans bibliothèque supplémentaire ; ils
+  sont comptés dans les Paramètres et affichés comme « (compressed journal entry) ». Les fichiers
+  texte tournés ou compressés (`*.1`, `*.gz`) ne sont pas lus.
+
 ## Export CSV
 
 Le bouton **Exporter** (à côté du nombre de logs) télécharge tous les logs stockés qui
@@ -247,6 +273,9 @@ résolutions.
 | `DOCKER_LOGS` | `on` dans compose | collecter les logs des conteneurs Docker locaux |
 | `DOCKER_HOST` | `tcp://docker-proxy:2375` dans compose | adresse de l'API Docker (`unix:///var/run/docker.sock` hors compose) |
 | `DOCKER_BACKFILL` | `1h` | historique lu pour un conteneur vu pour la première fois |
+| `HOST_LOGS_GID` | `4` (adm) dans compose | groupe donné au conteneur pour lire les logs de l'hôte |
+| `HOST_LOGS_BACKFILL` | `1h` | historique lu à l'activation de la source « logs système de l'hôte » |
+| `HOST_LOGS_ROOT` | `/host` | emplacement de montage des répertoires de l'hôte |
 | `TZ` | `Europe/Paris` | fuseau horaire des horodatages RFC 3164 (qui n'en portent pas) |
 | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | réglage de l'ingestion |
 
@@ -303,6 +332,7 @@ Pour mettre à jour l'une d'elles :
 | `export.go` | export CSV en flux |
 | `docker.go` | logs des conteneurs Docker (API, lecteurs, positions, détection du niveau) |
 | `syslogserver.go` | écoutes syslog ouvertes et fermées depuis Paramètres > Sources |
+| `hostlogs.go`, `journal.go` | logs système de l'hôte : lecteur du journal systemd (sans `journalctl`) et suivi de `/var/log` |
 | `tags.go` | stockage des tags de couleur |
 | `api.go` | routes HTTP `/api/*` |
 | `web/` | interface (HTML, CSS, JavaScript simple, sans étape de build), embarquée dans le binaire ; les traductions sont dans `web/app.js` (`I18N`) |

README.md

@@ -147,6 +147,32 @@ colored and exported like syslog messages:
 therefore goes through [docker-socket-proxy](https://github.com/Tecnativa/docker-socket-proxy),
 which only lets through listing containers, reading logs, events and engine info (`GET` only).
 
+## Host system logs
+
+Logstream can also collect the system logs of the machine hosting the stack, without
+configuring anything on the host. The source is **off by default**: turn it on in
+**Settings > Sources > Host system logs** (saved in `/data/hostlogs.json`).
+
+- `docker-compose.yml` mounts `/var/log` and `/run/log/journal` read-only under `/host`.
+- When the host runs systemd, Logstream reads the **systemd journal** files directly (no
+  `journalctl` needed in the image): **host** is the machine name, **app** the program
+  (`SYSLOG_IDENTIFIER`), severity and facility come from the journal, and the systemd unit is
+  kept in `unit`. Otherwise it follows the text files of `/var/log` (`syslog`, `messages`,
+  `*.log`), parsed like syslog lines, with the file name in `log_file`.
+- These logs have the `host` source: the **Source** filter shows them alone.
+- When the source is turned on, the last hour is read first (`HOST_LOGS_BACKFILL`); the
+  position reached is saved in `/data/hostlogs-state.json`, so a restart neither loses nor
+  duplicates entries.
+- **Permissions**: the container runs as an unprivileged user and gets the `adm` group
+  (gid 4), which can read the journal and `/var/log` on Debian and Ubuntu. On other systems,
+  set `HOST_LOGS_GID` in `.env` to the gid of `systemd-journal`
+  (`getent group systemd-journal | cut -d: -f3`). Settings > Sources shows a clear message when
+  access is denied.
+- Limits: journal fields compressed by journald (messages longer than 512 bytes, compressed
+  with zstd/lz4/xz) cannot be decoded without extra libraries; they are counted in Settings and
+  shown as "(compressed journal entry)". Rotated or compressed text files (`*.1`, `*.gz`) are
+  not read.
+
 ## CSV export
 
 The **Export** button (next to the log count) downloads every stored log matching the
@@ -222,6 +248,9 @@ are only known by your router or a local DNS (Pi-hole, AdGuard, Unbound…), set
 | `DOCKER_LOGS` | `on` in compose | collect the logs of the local Docker containers |
 | `DOCKER_HOST` | `tcp://docker-proxy:2375` in compose | Docker API address (`unix:///var/run/docker.sock` outside compose) |
 | `DOCKER_BACKFILL` | `1h` | history read from a container seen for the first time |
+| `HOST_LOGS_GID` | `4` (adm) in compose | group given to the container to read the host logs |
+| `HOST_LOGS_BACKFILL` | `1h` | history read when the host system logs source is turned on |
+| `HOST_LOGS_ROOT` | `/host` | where the host directories are mounted |
 | `TZ` | `Europe/Paris` | time zone for RFC 3164 timestamps (which carry none) |
 | `BATCH_SIZE`, `FLUSH_MS`, `QUEUE_SIZE` | `1000`, `1000`, `100000` | ingestion tuning |
 
@@ -275,6 +304,7 @@ To update one of them:
 | `export.go` | streamed CSV export |
 | `docker.go` | Docker container logs (API, followers, positions, level detection) |
 | `syslogserver.go` | syslog listeners opened and closed from Settings > Sources |
+| `hostlogs.go`, `journal.go` | host system logs: systemd journal reader (no `journalctl`) and `/var/log` follower |
 | `tags.go` | color tag storage |
 | `api.go` | `/api/*` HTTP routes |
 | `web/` | UI (HTML, CSS, plain JavaScript, no build step), embedded in the binary; translations live in `web/app.js` (`I18N`) |

api.go

@@ -21,6 +21,7 @@ type API struct {
 	exportMax  int
 	docker     *DockerManager // nil when DOCKER_LOGS is off
 	syslog     *SyslogServer
+	host       *HostLogs
 }
 
 func (a *API) Routes(mux *http.ServeMux) {
@@ -42,6 +43,28 @@ func (a *API) Routes(mux *http.ServeMux) {
 	mux.HandleFunc("PUT /api/syslog", a.syslogConfigure)
 	mux.HandleFunc("GET /api/docker", a.dockerStatus)
 	mux.HandleFunc("PUT /api/docker", a.dockerConfigure)
+	mux.HandleFunc("GET /api/hostlogs", a.hostLogsStatus)
+	mux.HandleFunc("PUT /api/hostlogs", a.hostLogsConfigure)
+}
+
+// GET /api/hostlogs: state of the host system logs source (Settings > Sources).
+func (a *API) hostLogsStatus(w http.ResponseWriter, r *http.Request) {
+	writeJSON(w, http.StatusOK, a.host.Status())
+}
+
+// PUT /api/hostlogs {"enabled": bool}
+func (a *API) hostLogsConfigure(w http.ResponseWriter, r *http.Request) {
+	var cfg hostLogsConfig
+	if err := json.NewDecoder(http.MaxBytesReader(w, r.Body, 4096)).Decode(&cfg); err != nil {
+		writeErr(w, http.StatusBadRequest, err)
+		return
+	}
+	if err := a.host.Configure(cfg); err != nil {
+		writeErr(w, http.StatusInternalServerError, err)
+		return
+	}
+	log.Printf("host system logs changed from %s: %+v", r.RemoteAddr, cfg)
+	writeJSON(w, http.StatusOK, a.host.Status())
 }
 
 // GET /api/syslog: syslog reception state (Settings > Sources).

docker-compose.yml

@@ -23,8 +23,14 @@ services:
       DOCKER_LOGS: ${DOCKER_LOGS:-on}             # collecte des logs des conteneurs Docker
       DOCKER_HOST: tcp://docker-proxy:2375        # lecture seul de l'API Docker
       DOCKER_BACKFILL: ${DOCKER_BACKFILL:-1h}
+      HOST_LOGS_BACKFILL: ${HOST_LOGS_BACKFILL:-1h}   # historique lu a l'activation des logs systeme de l'hote
+    group_add:
+      - "${HOST_LOGS_GID:-4}"        # groupe autorise a lire les logs de l'hote (4 = adm sur Debian/Ubuntu)
     volumes:
       - logstream-data:/data         # tags.json, docker.json (les choix des conteneurs)
+      # logs systeme de l'hote, en lecture seule (source a activer dans Reglages > Sources)
+      - /var/log:/host/var/log:ro                     # journal systemd persistant et fichiers texte
+      - /run/log/journal:/host/run/log/journal:ro     # journal systemd volatile
     labels:
       logstream.exclude: "true"      # pas de collect des logs logstream
 

hostlogs.go

@@ -0,0 +1,452 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"io"
+	"io/fs"
+	"log"
+	"os"
+	"path/filepath"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"syscall"
+	"time"
+	"unicode/utf8"
+)
+
+// hostLogsConfig is saved in /data/hostlogs.json and edited in Settings > Sources.
+type hostLogsConfig struct {
+	Enabled bool `json:"enabled"`
+}
+
+// hostPos is where reading resumes in a journal file (by file ID) or a text
+// file (by name, with its inode to detect rotations). Off -1: archived file
+// read to the end.
+type hostPos struct {
+	Off int64  `json:"off"`
+	Ino uint64 `json:"ino,omitempty"`
+}
+
+// HostLogs collects the system logs of the machine hosting the stack: the
+// systemd journal when there is one, else the text files of /var/log. The
+// host directories are mounted read-only under root (/host by default).
+type HostLogs struct {
+	root      string
+	sink      func(*Entry)
+	backfill  time.Duration
+	cfgPath   string
+	statePath string
+	wake      chan struct{}
+
+	// Owned by the Run goroutine.
+	pos     map[string]hostPos
+	dirty   bool
+	started bool // a first scan was done since enabling: new files are read from their start
+
+	mu         sync.Mutex
+	cfg        hostLogsConfig
+	reset      bool
+	mode       string // "journal", "files" or "" (nothing found)
+	files      int
+	read       uint64
+	compressed uint64
+	errCode    string
+	errDetail  string
+}
+
+func NewHostLogs(root, dataDir string, backfill time.Duration, sink func(*Entry)) *HostLogs {
+	h := &HostLogs{
+		root:      root,
+		sink:      sink,
+		backfill:  backfill,
+		cfgPath:   filepath.Join(dataDir, "hostlogs.json"),
+		statePath: filepath.Join(dataDir, "hostlogs-state.json"),
+		wake:      make(chan struct{}, 1),
+		pos:       map[string]hostPos{},
+	}
+	if b, err := os.ReadFile(h.cfgPath); err == nil {
+		_ = json.Unmarshal(b, &h.cfg)
+	}
+	if b, err := os.ReadFile(h.statePath); err == nil {
+		_ = json.Unmarshal(b, &h.pos)
+		h.started = len(h.pos) > 0
+	}
+	return h
+}
+
+// Run polls the host logs every second while the source is enabled.
+func (h *HostLogs) Run(ctx context.Context) {
+	tick := time.NewTicker(time.Second)
+	defer tick.Stop()
+	n := 0
+	for {
+		select {
+		case <-ctx.Done():
+			h.saveState()
+			return
+		case <-tick.C:
+		case <-h.wake:
+		}
+		h.mu.Lock()
+		enabled, reset := h.cfg.Enabled, h.reset
+		h.reset = false
+		h.mu.Unlock()
+		if reset {
+			// Disabled then enabled again: start over from HOST_LOGS_BACKFILL ago
+			// rather than reading everything written in between.
+			h.pos, h.started, h.dirty = map[string]hostPos{}, false, true
+		}
+		if enabled {
+			h.scan(time.Now())
+		}
+		if n++; n%5 == 0 || reset {
+			h.saveState()
+		}
+	}
+}
+
+func (h *HostLogs) saveState() {
+	if !h.dirty {
+		return
+	}
+	h.dirty = false
+	if err := writeJSONFile(h.statePath, h.pos); err != nil {
+		log.Printf("host logs: saving positions: %v", err)
+	}
+}
+
+func (h *HostLogs) setStatus(mode string, files int, code, detail string) {
+	h.mu.Lock()
+	h.mode, h.files, h.errCode, h.errDetail = mode, files, code, detail
+	h.mu.Unlock()
+}
+
+// scan reads what was added since the previous poll.
+func (h *HostLogs) scan(now time.Time) {
+	notBefore := time.Time{}
+	if !h.started {
+		notBefore = now.Add(-h.backfill)
+	}
+	defer func() { h.started = true }()
+
+	var journals []string
+	for _, dir := range []string{"var/log/journal", "run/log/journal"} {
+		base := filepath.Join(h.root, dir)
+		for _, pat := range []string{"*.journal", "*/*.journal"} {
+			m, _ := filepath.Glob(filepath.Join(base, pat))
+			journals = append(journals, m...)
+		}
+	}
+	if len(journals) > 0 {
+		h.scanJournals(journals, notBefore)
+		return
+	}
+	h.scanFiles(notBefore.IsZero())
+}
+
+func (h *HostLogs) scanJournals(paths []string, notBefore time.Time) {
+	seen := map[string]bool{}
+	var firstErr error
+	for _, p := range paths {
+		f, err := os.Open(p)
+		if err != nil {
+			firstErr = keepFirst(firstErr, err)
+			continue
+		}
+		hdr, err := readJournalHeader(f)
+		f.Close()
+		if err != nil {
+			continue // file being created, or not a journal
+		}
+		key := "j:" + hdr.fileID
+		seen[key] = true
+		pos, known := h.pos[key]
+		if known && pos.Off < 0 {
+			continue
+		}
+		from, nb := pos.Off, time.Time{}
+		if !known {
+			if hdr.archived && !notBefore.IsZero() && time.UnixMicro(int64(hdr.tailRealtimeUS)).Before(notBefore) {
+				h.pos[key], h.dirty = hostPos{Off: -1}, true // archived before the backfill window
+				continue
+			}
+			nb = notBefore
+		}
+		next, hdr, err := readJournal(p, from, nb, h.emitJournal)
+		if err != nil {
+			firstErr = keepFirst(firstErr, err)
+			continue
+		}
+		if hdr.archived {
+			next = -1
+		}
+		if !known || next != pos.Off {
+			h.pos[key], h.dirty = hostPos{Off: next}, true
+		}
+	}
+	h.prune("j:", seen)
+	code, detail := "", ""
+	if firstErr != nil && len(seen) == 0 {
+		code, detail = errCode(firstErr), firstErr.Error()
+	}
+	h.setStatus("journal", len(seen), code, detail)
+}
+
+func keepFirst(first, err error) error {
+	if first != nil {
+		return first
+	}
+	return err
+}
+
+func errCode(err error) string {
+	if errors.Is(err, fs.ErrPermission) {
+		return "permission"
+	}
+	return "read"
+}
+
+func (h *HostLogs) prune(prefix string, seen map[string]bool) {
+	for k := range h.pos {
+		if strings.HasPrefix(k, prefix) && !seen[k] {
+			delete(h.pos, k)
+			h.dirty = true
+		}
+	}
+}
+
+// hostLogFile reports the classic text logs of /var/log (rotated and
+// compressed copies are left out).
+func hostLogFile(name string) bool {
+	return name == "syslog" || name == "messages" || strings.HasSuffix(name, ".log")
+}
+
+const maxHostRead = 4 << 20 // per file and per poll
+
+// scanFiles follows the text files of /var/log, for hosts without journald.
+// Files present at the first scan are read from their end (only new lines).
+func (h *HostLogs) scanFiles(fromStart bool) {
+	dir := filepath.Join(h.root, "var/log")
+	ents, err := os.ReadDir(dir)
+	if err != nil {
+		code := errCode(err)
+		if errors.Is(err, fs.ErrNotExist) {
+			code = "not_mounted"
+		}
+		h.setStatus("", 0, code, err.Error())
+		return
+	}
+	seen := map[string]bool{}
+	var firstErr error
+	for _, de := range ents {
+		if !de.Type().IsRegular() || !hostLogFile(de.Name()) {
+			continue
+		}
+		name := de.Name()
+		key := "f:" + name
+		fi, err := de.Info()
+		if err != nil {
+			continue
+		}
+		var ino uint64
+		if st, ok := fi.Sys().(*syscall.Stat_t); ok {
+			ino = uint64(st.Ino)
+		}
+		pos, known := h.pos[key]
+		switch {
+		case !known && !fromStart:
+			pos = hostPos{Off: fi.Size(), Ino: ino}
+		case !known || pos.Ino != ino || fi.Size() < pos.Off:
+			pos = hostPos{Off: 0, Ino: ino} // new, rotated or truncated file
+		}
+		if fi.Size() > pos.Off {
+			off, err := h.readFile(filepath.Join(dir, name), name, pos.Off)
+			if err != nil {
+				firstErr = keepFirst(firstErr, err)
+				continue // not readable: not counted as followed
+			}
+			pos.Off = off
+		}
+		seen[key] = true
+		if old, ok := h.pos[key]; !ok || old != pos {
+			h.pos[key], h.dirty = pos, true
+		}
+	}
+	h.prune("f:", seen)
+	code, detail := "", ""
+	if firstErr != nil && len(seen) == 0 {
+		code, detail = errCode(firstErr), firstErr.Error()
+	}
+	mode := "files"
+	if len(seen) == 0 && code == "" {
+		mode, code = "", "empty"
+	}
+	h.setStatus(mode, len(seen), code, detail)
+}
+
+// readFile sends the complete lines written after off and returns the new offset.
+func (h *HostLogs) readFile(path, name string, off int64) (int64, error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return off, err
+	}
+	defer f.Close()
+	buf, err := io.ReadAll(io.NewSectionReader(f, off, maxHostRead))
+	if err != nil {
+		return off, err
+	}
+	end := bytes.LastIndexByte(buf, '\n')
+	if end < 0 {
+		if len(buf) < maxHostRead {
+			return off, nil // partial line: wait for its end
+		}
+		end = len(buf) - 1 // line longer than the read window: cut it
+	}
+	now := time.Now()
+	fac := fileFacility(name)
+	for _, line := range bytes.Split(buf[:end+1], []byte{'\n'}) {
+		if len(bytes.TrimSpace(line)) == 0 {
+			continue
+		}
+		e := ParseSyslog(line, "", "file", now)
+		if e.Host == "" {
+			e.Host = "localhost"
+		}
+		if n, ok := detectLevel(e.Message); ok {
+			e.SevNum, e.Severity = n, severityNames[n]
+		} else {
+			e.SevNum, e.Severity = 6, "info"
+		}
+		if fac >= 0 {
+			e.Facility = facilityNames[fac]
+		}
+		e.SourceType = "host"
+		e.Extra = map[string]string{"log_file": "/var/log/" + name}
+		h.sink(e)
+		h.count(1, 0)
+	}
+	return off + int64(end) + 1, nil
+}
+
+// fileFacility guesses the facility from the usual Debian/RHEL file names.
+func fileFacility(name string) int {
+	switch strings.TrimSuffix(name, ".log") {
+	case "kern":
+		return 0
+	case "mail", "maillog":
+		return 2
+	case "daemon":
+		return 3
+	case "auth", "secure":
+		return 4
+	case "cron":
+		return 9
+	}
+	return -1
+}
+
+func (h *HostLogs) count(read, compressed uint64) {
+	h.mu.Lock()
+	h.read += read
+	h.compressed += compressed
+	h.mu.Unlock()
+}
+
+// emitJournal turns a journal entry into an Entry.
+func (h *HostLogs) emitJournal(je *journalEntry) {
+	f := je.Fields
+	msg := f["MESSAGE"]
+	if msg == "" && je.Compressed > 0 {
+		msg = "(compressed journal entry: read it with journalctl)"
+	}
+	h.count(1, uint64(je.Compressed))
+	if strings.TrimSpace(msg) == "" {
+		return
+	}
+	if !utf8.ValidString(msg) {
+		msg = strings.ToValidUTF8(msg, "�")
+	}
+	sev := 6
+	if n, err := strconv.Atoi(f["PRIORITY"]); err == nil && n >= 0 && n <= 7 {
+		sev = n
+	}
+	fac := 1 // user
+	switch {
+	case f["_TRANSPORT"] == "kernel":
+		fac = 0
+	case f["_SYSTEMD_UNIT"] != "":
+		fac = 3 // daemon
+	}
+	if n, err := strconv.Atoi(f["SYSLOG_FACILITY"]); err == nil && n >= 0 && n < len(facilityNames) {
+		fac = n
+	}
+	app := firstNonEmpty(f["SYSLOG_IDENTIFIER"], f["_COMM"], strings.TrimSuffix(f["_SYSTEMD_UNIT"], ".service"))
+	host := firstNonEmpty(f["_HOSTNAME"], "localhost")
+	h.sink(&Entry{
+		Time:       je.Realtime,
+		Received:   time.Now(),
+		Host:       host,
+		App:        app,
+		ProcID:     firstNonEmpty(f["SYSLOG_PID"], f["_PID"]),
+		Facility:   facilityNames[fac],
+		Severity:   severityNames[sev],
+		SevNum:     sev,
+		Message:    strings.TrimRight(msg, "\n"),
+		Proto:      "journal",
+		SourceType: "host",
+		Extra:      map[string]string{"unit": f["_SYSTEMD_UNIT"]},
+	})
+}
+
+func firstNonEmpty(vals ...string) string {
+	for _, v := range vals {
+		if v != "" {
+			return v
+		}
+	}
+	return ""
+}
+
+// Configure saves and applies the configuration.
+func (h *HostLogs) Configure(cfg hostLogsConfig) error {
+	h.mu.Lock()
+	if h.cfg.Enabled && !cfg.Enabled {
+		h.reset = true
+		h.mode, h.files, h.errCode, h.errDetail = "", 0, "", ""
+	}
+	h.cfg = cfg
+	h.mu.Unlock()
+	select {
+	case h.wake <- struct{}{}:
+	default:
+	}
+	return writeJSONFile(h.cfgPath, cfg)
+}
+
+// Status is the state shown in Settings > Sources.
+func (h *HostLogs) Status() map[string]any {
+	h.mu.Lock()
+	defer h.mu.Unlock()
+	dirs := []string{}
+	for _, d := range []string{"var/log/journal", "run/log/journal", "var/log"} {
+		if _, err := os.Stat(filepath.Join(h.root, d)); err == nil {
+			dirs = append(dirs, "/"+d)
+		}
+	}
+	sort.Strings(dirs)
+	return map[string]any{
+		"enabled":    h.cfg.Enabled,
+		"mode":       h.mode,
+		"files":      h.files,
+		"read":       h.read,
+		"compressed": h.compressed,
+		"code":       h.errCode,
+		"error":      h.errDetail,
+		"mounted":    dirs,
+	}
+}

hostlogs_test.go

@@ -0,0 +1,206 @@
+package main
+
+import (
+	"encoding/binary"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+// fakeJournal builds a minimal journal file: header, data objects, entries.
+type fakeJournal struct {
+	compact bool
+	buf     []byte
+	seq     uint64
+	tail    uint64
+}
+
+func newFakeJournal(compact bool) *fakeJournal {
+	j := &fakeJournal{compact: compact, buf: make([]byte, 272)}
+	copy(j.buf, jSignature)
+	if compact {
+		binary.LittleEndian.PutUint32(j.buf[12:], jIncompatCompact)
+	}
+	j.buf[16] = 1 // online
+	copy(j.buf[24:40], "0123456789abcdef")
+	binary.LittleEndian.PutUint64(j.buf[88:], 272)
+	return j
+}
+
+func (j *fakeJournal) object(typ, flags byte, body []byte) uint64 {
+	for len(j.buf)%8 != 0 {
+		j.buf = append(j.buf, 0)
+	}
+	off := uint64(len(j.buf))
+	h := make([]byte, jObjHeaderSize)
+	h[0], h[1] = typ, flags
+	binary.LittleEndian.PutUint64(h[8:], uint64(jObjHeaderSize+len(body)))
+	j.buf = append(append(j.buf, h...), body...)
+	j.tail = off
+	binary.LittleEndian.PutUint64(j.buf[136:], off)
+	return off
+}
+
+func (j *fakeJournal) data(field string, flags byte) uint64 {
+	n := 48
+	if j.compact {
+		n = 56
+	}
+	return j.object(jObjData, flags, append(make([]byte, n), field...))
+}
+
+// entry appends an entry; linked=false leaves it unfinished (tail seqnum not updated).
+func (j *fakeJournal) entry(t time.Time, linked bool, fields ...string) {
+	var items []byte
+	for _, f := range fields {
+		flags := byte(0)
+		if strings.HasPrefix(f, "!") { // compressed data object
+			f, flags = f[1:], 4
+		}
+		off := j.data(f, flags)
+		if j.compact {
+			items = binary.LittleEndian.AppendUint32(items, uint32(off))
+		} else {
+			items = binary.LittleEndian.AppendUint64(items, off)
+			items = binary.LittleEndian.AppendUint64(items, 0)
+		}
+	}
+	j.seq++
+	body := make([]byte, 48)
+	binary.LittleEndian.PutUint64(body[0:], j.seq)
+	binary.LittleEndian.PutUint64(body[8:], uint64(t.UnixMicro()))
+	j.object(jObjEntry, 0, append(body, items...))
+	if linked {
+		binary.LittleEndian.PutUint64(j.buf[160:], j.seq)
+		binary.LittleEndian.PutUint64(j.buf[192:], uint64(t.UnixMicro()))
+	}
+}
+
+func (j *fakeJournal) write(t *testing.T, path string) {
+	t.Helper()
+	if err := os.WriteFile(path, j.buf, 0o644); err != nil {
+		t.Fatal(err)
+	}
+}
+
+func TestReadJournal(t *testing.T) {
+	for _, compact := range []bool{false, true} {
+		path := filepath.Join(t.TempDir(), "system.journal")
+		now := time.Now()
+		j := newFakeJournal(compact)
+		j.entry(now.Add(-2*time.Hour), true, "MESSAGE=too old", "PRIORITY=6")
+		j.entry(now.Add(-time.Minute), true, "MESSAGE=Started cron.", "PRIORITY=5", "SYSLOG_IDENTIFIER=systemd", "_HOSTNAME=srv1", "_PID=1")
+		j.write(t, path)
+
+		var got []*journalEntry
+		emit := func(e *journalEntry) { got = append(got, e) }
+		next, _, err := readJournal(path, 0, now.Add(-time.Hour), emit)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "Started cron." || got[0].Fields["_HOSTNAME"] != "srv1" {
+			t.Fatalf("compact=%v: got %+v", compact, got)
+		}
+
+		// A new complete entry and one still being written.
+		j.entry(now, true, "MESSAGE=second", "!MESSAGE=big")
+		j.entry(now, false, "MESSAGE=unfinished")
+		j.write(t, path)
+		got = nil
+		next2, _, err := readJournal(path, next, time.Time{}, emit)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "second" || got[0].Compressed != 1 {
+			t.Fatalf("compact=%v: resumed read got %d entries", compact, len(got))
+		}
+
+		// Once linked, the unfinished entry is read from where reading stopped.
+		binary.LittleEndian.PutUint64(j.buf[160:], j.seq)
+		j.write(t, path)
+		got = nil
+		if _, _, err := readJournal(path, next2, time.Time{}, emit); err != nil {
+			t.Fatal(err)
+		}
+		if len(got) != 1 || got[0].Fields["MESSAGE"] != "unfinished" {
+			t.Fatalf("compact=%v: unfinished entry got %+v", compact, got)
+		}
+	}
+}
+
+func TestHostLogsJournal(t *testing.T) {
+	root := t.TempDir()
+	dir := filepath.Join(root, "var/log/journal/machine")
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	j := newFakeJournal(true)
+	j.entry(time.Now(), true, "MESSAGE=Accepted publickey", "PRIORITY=6", "SYSLOG_FACILITY=10", "SYSLOG_IDENTIFIER=sshd", "_PID=42", "_HOSTNAME=srv1", "_SYSTEMD_UNIT=ssh.service")
+	j.entry(time.Now(), true, "MESSAGE=oops", "PRIORITY=3", "_TRANSPORT=kernel", "_HOSTNAME=srv1")
+	j.write(t, filepath.Join(dir, "system.journal"))
+
+	var got []*Entry
+	h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) })
+	h.scan(time.Now())
+	if len(got) != 2 {
+		t.Fatalf("got %d entries", len(got))
+	}
+	e := got[0]
+	if e.App != "sshd" || e.ProcID != "42" || e.Facility != "authpriv" || e.Severity != "info" || e.Host != "srv1" || e.SourceType != "host" || e.Extra["unit"] != "ssh.service" {
+		t.Fatalf("entry %+v", e)
+	}
+	if got[1].Facility != "kern" || got[1].Severity != "err" {
+		t.Fatalf("kernel entry %+v", got[1])
+	}
+	h.scan(time.Now()) // nothing new
+	if len(got) != 2 {
+		t.Fatalf("re-read: %d entries", len(got))
+	}
+	if st := h.Status(); st["mode"] != "journal" || st["files"] != 1 {
+		t.Fatalf("status %+v", st)
+	}
+}
+
+func TestHostLogsFiles(t *testing.T) {
+	root := t.TempDir()
+	dir := filepath.Join(root, "var/log")
+	if err := os.MkdirAll(dir, 0o755); err != nil {
+		t.Fatal(err)
+	}
+	auth := filepath.Join(dir, "auth.log")
+	if err := os.WriteFile(auth, []byte("Oct  3 07:00:00 srv1 sshd[1]: old line\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	_ = os.WriteFile(filepath.Join(dir, "auth.log.1"), []byte("rotated\n"), 0o644)
+
+	var got []*Entry
+	h := NewHostLogs(root, t.TempDir(), time.Hour, func(e *Entry) { got = append(got, e) })
+	h.scan(time.Now()) // existing content is skipped
+	if len(got) != 0 {
+		t.Fatalf("first scan read %d lines", len(got))
+	}
+	f, _ := os.OpenFile(auth, os.O_APPEND|os.O_WRONLY, 0)
+	_, _ = f.WriteString("Oct  3 08:00:00 srv1 sshd[7]: Failed password for root\nOct  3 08:00:01 srv1 sshd[7]: partial")
+	f.Close()
+	h.scan(time.Now())
+	if len(got) != 1 {
+		t.Fatalf("got %d lines", len(got))
+	}
+	e := got[0]
+	if e.Host != "srv1" || e.App != "sshd" || e.ProcID != "7" || e.Facility != "auth" || e.Extra["log_file"] != "/var/log/auth.log" || e.Proto != "file" {
+		t.Fatalf("entry %+v", e)
+	}
+	if st := h.Status(); st["mode"] != "files" || st["files"] != 1 {
+		t.Fatalf("status %+v", st)
+	}
+}
+
+func TestHostLogsNotMounted(t *testing.T) {
+	h := NewHostLogs(filepath.Join(t.TempDir(), "none"), t.TempDir(), time.Hour, func(*Entry) {})
+	h.scan(time.Now())
+	if st := h.Status(); st["code"] != "not_mounted" {
+		t.Fatalf("status %+v", st)
+	}
+}

journal.go

@@ -0,0 +1,174 @@
+package main
+
+import (
+	"bufio"
+	"bytes"
+	"encoding/binary"
+	"encoding/hex"
+	"errors"
+	"io"
+	"os"
+	"time"
+)
+
+// Minimal reader of systemd journal files (*.journal), enough to follow them
+// without journalctl: objects are walked in file order, which is the order
+// entries were written. Format: https://systemd.io/JOURNAL_FILE_FORMAT/
+
+const (
+	jHeaderMin     = 208 // header fields used below end at offset 208
+	jObjHeaderSize = 16
+
+	jObjData  = 1
+	jObjEntry = 3
+
+	jIncompatCompact = 1 << 4
+	jStateArchived   = 2
+	jObjCompressed   = 1<<0 | 1<<1 | 1<<2 // XZ, LZ4, ZSTD: not decoded (no stdlib codec)
+)
+
+var jSignature = []byte("LPKSHHRH")
+
+// jHeader holds the header fields the reader needs.
+type jHeader struct {
+	compact        bool
+	archived       bool
+	fileID         string
+	headerSize     uint64
+	tailObject     uint64 // offset of the last object
+	tailSeqnum     uint64 // seqnum of the last complete entry
+	tailRealtimeUS uint64 // realtime of the last entry (µs since the epoch)
+}
+
+func readJournalHeader(f io.ReaderAt) (jHeader, error) {
+	b := make([]byte, jHeaderMin)
+	if _, err := f.ReadAt(b, 0); err != nil {
+		return jHeader{}, err
+	}
+	if !bytes.Equal(b[:8], jSignature) {
+		return jHeader{}, errors.New("not a journal file")
+	}
+	le := binary.LittleEndian
+	h := jHeader{
+		compact:        le.Uint32(b[12:])&jIncompatCompact != 0,
+		archived:       b[16] == jStateArchived,
+		fileID:         hex.EncodeToString(b[24:40]),
+		headerSize:     le.Uint64(b[88:]),
+		tailObject:     le.Uint64(b[136:]),
+		tailSeqnum:     le.Uint64(b[160:]),
+		tailRealtimeUS: le.Uint64(b[192:]),
+	}
+	if h.headerSize < jHeaderMin {
+		return h, errors.New("journal header too small")
+	}
+	return h, nil
+}
+
+// journalEntry is one entry: its time and its "FIELD=value" pairs.
+type journalEntry struct {
+	Realtime   time.Time
+	Fields     map[string]string
+	Compressed int // fields that could not be read (compressed data)
+}
+
+// readJournal reads the entries complete after offset `from` (0 = start of
+// the file) and returns the offset to resume from. Entries older than
+// `notBefore` are skipped without decoding their fields.
+func readJournal(path string, from int64, notBefore time.Time, emit func(*journalEntry)) (next int64, h jHeader, err error) {
+	f, err := os.Open(path)
+	if err != nil {
+		return from, h, err
+	}
+	defer f.Close()
+	if h, err = readJournalHeader(f); err != nil {
+		return from, h, err
+	}
+	if from < int64(h.headerSize) {
+		from = int64(h.headerSize)
+	}
+	end := int64(h.tailObject)
+	r := bufio.NewReaderSize(io.NewSectionReader(f, from, 1<<62), 64*1024)
+	le := binary.LittleEndian
+	hdr := make([]byte, jObjHeaderSize)
+	off := from
+	for off <= end {
+		if _, err := io.ReadFull(r, hdr); err != nil {
+			return off, h, nil // object still being written
+		}
+		typ, size := hdr[0], int64(le.Uint64(hdr[8:]))
+		if size < jObjHeaderSize {
+			return off, h, nil
+		}
+		body := size - jObjHeaderSize
+		if typ == jObjEntry && body >= 48 && body < 1<<20 {
+			obj := make([]byte, body)
+			if _, err := io.ReadFull(r, obj); err != nil {
+				return off, h, nil
+			}
+			seq := le.Uint64(obj[0:])
+			if seq == 0 || seq > h.tailSeqnum {
+				return off, h, nil // not linked yet: retry at the next poll
+			}
+			rt := time.UnixMicro(int64(le.Uint64(obj[8:])))
+			if !rt.Before(notBefore) {
+				emit(readEntryFields(f, h.compact, rt, obj[48:]))
+			}
+		} else if _, err := r.Discard(int(body)); err != nil {
+			return off, h, nil
+		}
+		next := (off + size + 7) &^ 7 // objects are 8-byte aligned
+		if pad := next - off - size; pad > 0 {
+			if _, err := r.Discard(int(pad)); err != nil {
+				return next, h, nil // the object is complete: resume after it
+			}
+		}
+		off = next
+	}
+	return off, h, nil
+}
+
+// readEntryFields resolves the data objects referenced by an entry.
+func readEntryFields(f io.ReaderAt, compact bool, rt time.Time, items []byte) *journalEntry {
+	le := binary.LittleEndian
+	e := &journalEntry{Realtime: rt, Fields: make(map[string]string, 16)}
+	step := 16
+	if compact {
+		step = 4
+	}
+	payloadAt := int64(64)
+	if compact {
+		payloadAt = 72
+	}
+	hdr := make([]byte, jObjHeaderSize)
+	for i := 0; i+step <= len(items); i += step {
+		var off int64
+		if compact {
+			off = int64(le.Uint32(items[i:]))
+		} else {
+			off = int64(le.Uint64(items[i:]))
+		}
+		if off == 0 {
+			continue
+		}
+		if _, err := f.ReadAt(hdr, off); err != nil || hdr[0] != jObjData {
+			continue
+		}
+		if hdr[1]&jObjCompressed != 0 {
+			e.Compressed++
+			continue
+		}
+		size := int64(le.Uint64(hdr[8:]))
+		n := size - payloadAt
+		if n <= 0 || n > 1<<20 {
+			continue
+		}
+		p := make([]byte, n)
+		if _, err := f.ReadAt(p, off+payloadAt); err != nil {
+			continue
+		}
+		if k := bytes.IndexByte(p, '='); k > 0 {
+			e.Fields[string(p[:k])] = string(p[k+1:])
+		}
+	}
+	return e
+}

main.go

@@ -1,4 +1,4 @@
-// Logstream: a syslog (UDP/TCP) sink stored in VictoriaLogs,
+// Logstream: a syslog (UDP/TCP), Docker and host system logs sink stored in VictoriaLogs,
 // with a web interface to browse and search the logs.
 package main
 
@@ -38,6 +38,8 @@ type config struct {
 	dockerLogs bool
 	dockerHost string
 	backfill   time.Duration
+	hostRoot   string
+	hostFill   time.Duration
 	batchSize  int
 	queueSize  int
 	flushEvery time.Duration
@@ -89,6 +91,8 @@ func main() {
 		dockerLogs: getenvBool("DOCKER_LOGS", false),
 		dockerHost: getenv("DOCKER_HOST", "unix:///var/run/docker.sock"),
 		backfill:   getenvDuration("DOCKER_BACKFILL", time.Hour),
+		hostRoot:   getenv("HOST_LOGS_ROOT", "/host"),
+		hostFill:   getenvDuration("HOST_LOGS_BACKFILL", time.Hour),
 		batchSize:  getenvInt("BATCH_SIZE", 1000),
 		queueSize:  getenvInt("QUEUE_SIZE", 100000),
 		flushEvery: time.Duration(getenvInt("FLUSH_MS", 1000)) * time.Millisecond,
@@ -139,6 +143,9 @@ func main() {
 			log.Printf("docker logs enabled through %s", cfg.dockerHost)
 		}
 	}
+	// System logs of the host (journal or /var/log), off until enabled in Settings > Sources.
+	api.host = NewHostLogs(cfg.hostRoot, cfg.dataDir, cfg.hostFill, sink)
+	go api.host.Run(ctx)
 	api.Routes(mux)
 	mux.Handle("GET /", http.FileServer(http.FS(static)))
 

query.go

@@ -17,7 +17,7 @@ type Filter struct {
 	Host     string
 	App      string
 	Severity int    // highest severity number included (0 = emerg … 7 = debug), -1 = all
-	Source   string // "syslog", "docker" or "" for all
+	Source   string // "syslog", "docker", "host" or "" for all
 }
 
 var rangeDurations = map[string]time.Duration{
@@ -119,10 +119,10 @@ func (f Filter) filterExpr() string {
 		parts = append(parts, "app:="+strconv.Quote(f.App))
 	}
 	switch f.Source {
-	case "docker":
+	case "docker", "host":
-		parts = append(parts, `source_type:="docker"`)
+		parts = append(parts, `source_type:=`+strconv.Quote(f.Source))
 	case "syslog": // also matches logs stored before source_type existed
-		parts = append(parts, `!(source_type:="docker")`)
+		parts = append(parts, `!(source_type:in("docker","host"))`)
 	}
 	if f.Severity >= 0 && f.Severity < 7 {
 		names := make([]string, 0, 8)
@@ -209,7 +209,7 @@ func (m *Matcher) Match(e *Entry) bool {
 	if m.f.App != "" && e.App != m.f.App {
 		return false
 	}
-	if (m.f.Source == "docker") != (e.SourceType == "docker") && m.f.Source != "" {
+	if m.f.Source != "" && m.f.Source != e.SourceType {
 		return false
 	}
 	if m.f.Severity >= 0 && e.SevNum > m.f.Severity {

web/app.js

@@ -67,6 +67,19 @@ const I18N = {
     fHostName: 'host name (DNS)', fHostIp: 'host IP',
     clickHost: 'Click to filter on this host', clickApp: 'Click to filter on this app',
     sourceAria: 'Source', srcAll: 'All sources', tabSources: 'Sources',
+    srcHost: 'Host system',
+    hostTitle: 'Host system logs',
+    hostEnabled: 'Collect the system logs of this machine',
+    hostOff: 'Off: the system logs of the machine hosting Logstream are not collected.',
+    hostWaiting: 'Starting…',
+    hostJournal: ({ n, r }) => `Reading the systemd journal (${n} files): ${r} entries since startup.`,
+    hostFiles: ({ n, r }) => `Following ${n} files of /var/log: ${r} lines since startup.`,
+    hostCompressed: (n) => ` ${n} compressed fields skipped (long messages, readable with journalctl).`,
+    host_not_mounted: 'No host log directory found: mount /var/log and /run/log/journal under /host (see docker-compose.yml).',
+    host_permission: 'Permission denied: give the container a group allowed to read the logs (HOST_LOGS_GID in .env, see the README). ',
+    host_empty: 'No systemd journal and no log file found in /var/log.',
+    host_read: 'Cannot read the host logs: ',
+    hostHelp: 'Reads the systemd journal of the host (/var/log/journal, /run/log/journal), or the text files of /var/log (syslog, messages, *.log) when there is no journal. Directories are mounted read-only. When turned on, the last hour is read first (HOST_LOGS_BACKFILL).',
     syslogEnabled: 'Receive syslog messages', syslogProtocols: 'Protocols', syslogPort: 'Port',
     syslogListening: ({ port, protos }) => `Listening on port ${port} (${protos}).`,
     syslogOff: 'Syslog reception is off: syslog messages are ignored (Docker logs are still collected).',
@@ -88,6 +101,7 @@ const I18N = {
     dockerHelp: 'Logs are read through docker-socket-proxy, a read-only gateway: Logstream can list containers and read their logs, nothing else. Choices apply per compose service (or container name), so they survive container re-creations.',
     fContainer: 'container', fContainerId: 'container ID', fImage: 'image', fProject: 'compose project',
     fService: 'compose service', fStream: 'stream', fSourceType: 'source',
+    fUnit: 'systemd unit', fLogFile: 'log file',
     export: 'Export', exportCsvHint: 'Comma separated, UTF-8',
     exportExcel: 'CSV for Excel', exportExcelHint: 'Semicolon separated, for Excel in French',
     exportNote: (n) => `Up to ${n} logs matching the current filters, newest first. Dates use the time zone chosen in Settings.`,
@@ -196,6 +210,19 @@ const I18N = {
     fHostName: 'nom d\'hôte (DNS)', fHostIp: 'IP de l\'hôte',
     clickHost: 'Cliquer pour filtrer sur cet hôte', clickApp: 'Cliquer pour filtrer sur cette appli',
     sourceAria: 'Source', srcAll: 'Toutes les sources', tabSources: 'Sources',
+    srcHost: 'Système hôte',
+    hostTitle: 'Logs système de l\'hôte',
+    hostEnabled: 'Collecter les logs système de cette machine',
+    hostOff: 'Désactivé : les logs système de la machine qui héberge Logstream ne sont pas collectés.',
+    hostWaiting: 'Démarrage…',
+    hostJournal: ({ n, r }) => `Lecture du journal systemd (${n} fichiers) : ${r} entrées depuis le démarrage.`,
+    hostFiles: ({ n, r }) => `Suivi de ${n} fichiers de /var/log : ${r} lignes depuis le démarrage.`,
+    hostCompressed: (n) => ` ${n} champs compressés ignorés (messages longs, lisibles avec journalctl).`,
+    host_not_mounted: 'Aucun répertoire de logs de l\'hôte trouvé : montez /var/log et /run/log/journal sous /host (voir docker-compose.yml).',
+    host_permission: 'Accès refusé : donnez au conteneur un groupe autorisé à lire les logs (HOST_LOGS_GID dans .env, voir le README). ',
+    host_empty: 'Ni journal systemd ni fichier de log trouvé dans /var/log.',
+    host_read: 'Lecture des logs de l\'hôte impossible : ',
+    hostHelp: 'Lit le journal systemd de l\'hôte (/var/log/journal, /run/log/journal), ou les fichiers texte de /var/log (syslog, messages, *.log) s\'il n\'y a pas de journal. Les répertoires sont montés en lecture seule. À l\'activation, la dernière heure est lue d\'abord (HOST_LOGS_BACKFILL).',
     syslogEnabled: 'Recevoir les messages syslog', syslogProtocols: 'Protocoles', syslogPort: 'Port',
     syslogListening: ({ port, protos }) => `Écoute sur le port ${port} (${protos}).`,
     syslogOff: 'Réception syslog désactivée : les messages syslog sont ignorés (les logs Docker sont toujours collectés).',
@@ -217,6 +244,7 @@ const I18N = {
     dockerHelp: 'Les logs sont lus via docker-socket-proxy, une passerelle en lecture seule : Logstream peut lister les conteneurs et lire leurs logs, rien d\'autre. Les choix s\'appliquent par service compose (ou nom de conteneur), ils survivent donc à la recréation des conteneurs.',
     fContainer: 'conteneur', fContainerId: 'ID du conteneur', fImage: 'image', fProject: 'projet compose',
     fService: 'service compose', fStream: 'flux', fSourceType: 'source',
+    fUnit: 'unité systemd', fLogFile: 'fichier de log',
     export: 'Exporter', exportCsvHint: 'Séparateur virgule, UTF-8',
     exportExcel: 'CSV pour Excel', exportExcelHint: 'Séparateur point-virgule, pour Excel en français',
     exportNote: (n) => `Jusqu'à ${n} logs correspondant aux filtres, du plus récent au plus ancien. Dates dans le fuseau choisi dans Paramètres.`,
@@ -447,6 +475,7 @@ function setLang(next) {
   renderPurge();
   renderInterface();
   renderSyslog();
+  renderHost();
   renderDocker();
 }
 
@@ -805,14 +834,14 @@ function updateCount(tookMs) {
 }
 
 function detailsHTML(r) {
-  const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg'];
+  const order = ['received', 'msg_time', '_time', 'host', 'host_name', 'host_ip', 'source_type', 'container', 'image', 'compose_project', 'compose_service', 'stream', 'unit', 'log_file', 'app', 'procid', 'severity', 'facility', 'source', 'proto', 'sd', '_msg'];
   const hidden = new Set(['_stream_id', '_stream', 'sevnum']);
   if (r.msg_time) hidden.add('_time'); // same as the reception time
   const keys = order.filter((k) => r[k] != null && r[k] !== '' && !hidden.has(k))
     .concat(Object.keys(r).filter((k) => !order.includes(k) && !hidden.has(k)).sort());
   const label = {
     container: t('fContainer'), container_id: t('fContainerId'), image: t('fImage'), compose_project: t('fProject'),
-    compose_service: t('fService'), stream: t('fStream'), source_type: t('fSourceType'),
+    compose_service: t('fService'), stream: t('fStream'), source_type: t('fSourceType'), unit: t('fUnit'), log_file: t('fLogFile'),
     msg_time: t('fTime'), host_name: t('fHostName'), host_ip: t('fHostIp'), received: t('fReceived'), _time: t('fTime'), _msg: t('fMsg'), procid: t('fPid'), sd: t('fSd') };
   const val = (k) => (k === '_time' || k === 'received' || k === 'msg_time'
     ? `${esc(fmtFull(r[k]))} <span class="muted">(${esc(r[k])})</span>`
@@ -1726,6 +1755,48 @@ $('#syslogProtos').addEventListener('click', (ev) => {
   configureSyslog({ [b.dataset.proto]: !syslogState[b.dataset.proto] });
 });
 
+/* ================= Host system logs ================= */
+
+let hostState = null;
+
+async function loadHost() {
+  try { hostState = await api('/api/hostlogs'); } catch { hostState = null; }
+  renderHost();
+}
+
+function renderHost() {
+  const s = hostState;
+  const status = $('#hostStatus');
+  if (!s) { status.textContent = ''; return; }
+  $('#hostEnabled').checked = s.enabled;
+  let text = t('hostWaiting');
+  let cls = 'muted';
+  if (!s.enabled) {
+    text = t('hostOff');
+  } else if (s.code) {
+    text = t('host_' + s.code) + (s.code === 'read' || s.code === 'permission' ? s.error || '' : '');
+    cls = 'bad';
+  } else if (s.mode === 'journal') {
+    text = t('hostJournal', { n: s.files, r: fmtNum(s.read) }) + (s.compressed ? t('hostCompressed', fmtNum(s.compressed)) : '');
+    cls = 'good';
+  } else if (s.mode === 'files') {
+    text = t('hostFiles', { n: s.files, r: fmtNum(s.read) });
+    cls = 'good';
+  }
+  status.textContent = text;
+  status.className = 'docker-status ' + cls;
+}
+
+async function configureHost(enabled) {
+  hostState = { ...(hostState || {}), enabled };
+  renderHost();
+  try { hostState = await api('/api/hostlogs', { method: 'PUT', body: { enabled } }); } catch (e) { toast(e.message); }
+  renderHost();
+  setTimeout(loadHost, 1500); // the first scan runs in the background
+}
+
+$('#hostEnabled').addEventListener('change', (ev) => configureHost(ev.target.checked));
+
 /* ================= Docker source ================= */
 
 let dockerState = null;
@@ -1839,7 +1910,10 @@ for (const [id, on] of [['#dockerAll', true], ['#dockerNone', false]]) {
 }
 // Keep the list fresh while the Sources tab is open.
 setInterval(() => {
-  if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) loadDocker();
+  if ($('#settingsDlg').open && !document.querySelector('[data-panel="sources"]').hidden) {
+    loadDocker();
+    loadHost();
+  }
 }, 4000);
 
 /* ================= Purge ================= */
@@ -2002,6 +2076,7 @@ $('#settingsBtn').addEventListener('click', () => {
   renderInterface();
   loadPurgeStatus();
   loadSyslog();
+  loadHost();
   loadDocker();
   showSettingsTab(store.get('settingsTab', 'locale'));
   $('#settingsDlg').showModal();

web/index.html

@@ -70,6 +70,7 @@
       <option value="" data-i18n="srcAll">All sources</option>
       <option value="syslog">Syslog</option>
       <option value="docker">Docker</option>
+      <option value="host" data-i18n="srcHost">Host system</option>
     </select>
     <span class="spacer"></span>
     <span id="count" class="muted"></span>
@@ -209,6 +210,16 @@
               </div>
             </section>
 
+            <section class="set-section">
+              <h3 data-i18n="hostTitle">Host system logs</h3>
+              <p id="hostStatus" class="docker-status"></p>
+              <label class="switch-row">
+                <input id="hostEnabled" type="checkbox" class="switch">
+                <span data-i18n="hostEnabled">Collect the system logs of this machine</span>
+              </label>
+              <p class="muted small" data-i18n="hostHelp"></p>
+            </section>
+
             <section class="set-section">
               <h3 data-i18n="dockerTitle">Docker containers</h3>
               <p id="dockerStatus" class="docker-status"></p>