vLab-BZH/pve-confsync
0
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: vLab-BZH/pve-confsync:phase-1-ui
vLab-BZH/pve-confsync:main vLab-BZH/pve-confsync:phase-1-ui
..
pull from: vLab-BZH/pve-confsync:main
vLab-BZH/pve-confsync:main vLab-BZH/pve-confsync:phase-1-ui
13 Commits
phase-1-ui .. main
Author SHA1 Message Date
cedricandClaude Opus 5.5 1e667ce8b9 Réplication de nœuds : réseau et DNS par l'API, multipath, chrony, sysctl par SSH 
Deuxième mode de réplication, séparé de la réplication cluster vers cluster
(sélecteur « Clusters (API) | Nœuds (fichiers) » en haut de page) : un profil
réplique la configuration système d'un nœud maître vers des nœuds cibles.

- réseau par /nodes/{node}/network (interfaces.new puis rechargement), adresses
  et passerelles propres à chaque nœud, mapping des noms d'interfaces par cible ;
  DNS par /nodes/{node}/dns ; multipath.conf, wwids (ajout seul), chrony,
  sysctl.d, interfaces.d (hors sdn) par SSH
- découverte des nœuds (/cluster/status), accès SSH par nœud (mot de passe du
  cluster ou propre), clé d'hôte épinglée, contrôle du nom du nœud
- dry-run avec diff des fichiers ; apply (admin) avec contrôle SHA-256,
  sauvegarde et restore.sh sur le nœud, écriture atomique, rechargements,
  restauration si un rechargement échoue, timer systemd de restauration si le
  nœud est injoignable après le rechargement du réseau, confirmation tapée
- vue Nœuds et profils, journal séparé par mode, démo avec nœuds simulés

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-23 20:29:55 +02:00
cedricandClaude Opus 5.5 dc1a7ff40e UI : badge de statut texte (À JOUR, ÉCARTS, BLOQUANT) sur les cartes à la place du liseré coloré 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-23 19:49:58 +02:00
cedricandClaude Opus 5.5 4980cd4c7f UI : éléments répliqués choisis dans la fenêtre de dry-run, menu de la barre retiré 
Le choix Répliquer / Ne pas répliquer de la fenêtre de dry-run devient la
liste enregistrée des éléments répliqués (administrateurs) : tous les
éléments y figurent, avec leurs dépendances, et un changement relance le
dry-run. Le menu « Éléments répliqués » de la barre du haut et l'exclusion
par exécution (skip) côté interface sont retirés.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-23 19:36:23 +02:00
cedricandClaude Opus 5.5 993fe2d8a7 UI : listes déroulantes lisibles en thème sombre, cadre unique sur la cible sélectionnée 
- option/optgroup prennent les couleurs du thème (texte clair sur fond clair sinon)
- carte cluster : bord de statut en ombre interne (suit les coins arrondis),
  sélection en contour accent de 2 px au lieu du double cadre

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
 2026-10-23 19:16:13 +02:00
claude Bot 2ca2f87307 UI : vocabulaire unifié, comptes + mots de passe, choix par élément, clé AES auto 2026-10-23 18:47:04 +02:00
claude Bot ee8343c69f Clusters : port 8006 par défaut, liste des cibles jamais null 2026-10-23 18:41:59 +02:00
claude Bot 8dd2f187ee Clusters : port 8006 par défaut, liste des cibles jamais null 2026-10-23 18:41:18 +02:00
claude Bot d24444726e README : création de la clé secrets/confsync_key avant docker compose up 2026-10-23 18:41:02 +02:00
cedric 4826fa5b82 Full web : fichiers restants 2026-10-23 17:43:20 +02:00
claude Bot ec3b7c9a01 Full web : clusters gérés dans l'UI, primaire et cibles, comptes pve 2026-10-23 17:23:14 +02:00
claude Bot 3e9cb563be Full web : clusters gérés dans l'UI, primaire et cibles, comptes pve 2026-10-23 17:20:16 +02:00
claude BotandClaude Opus 5.5 0fb88cb4b0 .env.example : variables de l'interface web, OIDC et CLI 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015a8P6WMRPeqdpd5VCh6jAy
 2026-10-14 08:59:39 +02:00
claude BotandClaude Opus 5.5 664e43a181 docker : compose de démo (serve -demo, sans secret ni volume) 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015a8P6WMRPeqdpd5VCh6jAy
 2026-10-13 20:58:38 +02:00
58 changed files with 8569 additions and 1736 deletions
Expand all files Collapse all files

No files matched your search

.env.example
+46
View File
@@ -0,0 +1,46 @@
# Copier en .env (ignoré par Git) puis adapter : cp .env.example .env
# Lu par docker compose (substitution ${...} et env_file des services) ; hors
# conteneur : set -a; . ./.env; set +a
# Les lignes commentées donnent la valeur par défaut. Pour les secrets, préférer la
# variante _FILE (secret docker monté dans /run/secrets) à une valeur en clair ici.
# ---------- Commun ----------
TZ=Europe/Paris
# Port publié sur l'hôte (docker-compose.yml et docker-compose.demo.yml)
# CONFSYNC_PORT=8080
# ---------- Interface web ----------
# LISTEN=:8080
# DATA_DIR=/data # base SQLite, secret.key, session.key
# SESSION_TTL=12h # durée Go : 30m, 8h, 24h…
# CONFSYNC_DEMO=false # true : faux clusters PVE, base temporaire
# Clé AES-256 des mots de passe root en base (64 caractères hex ou base64 de 32 octets) :
# openssl rand -hex 32
# Ordre de lecture : CONFSYNC_SECRET_KEY, puis CONFSYNC_SECRET_KEY_FILE, sinon DATA_DIR/secret.key
# (créée automatiquement au 1er démarrage). Une valeur CONFSYNC_SECRET_KEY ici est prioritaire. Sans cette clé, les mots de passe enregistrés sont perdus.
# CONFSYNC_SECRET_KEY=
# CONFSYNC_SECRET_KEY_FILE=/run/secrets/confsync_key
# Premier administrateur, créé au démarrage seulement si la base n'a aucun utilisateur
# et qu'un mot de passe est fourni. Sinon, la page de connexion le fait créer : le premier
# qui atteint l'instance devient administrateur.
# CONFSYNC_ADMIN_USER=admin
# CONFSYNC_ADMIN_PASSWORD=
# CONFSYNC_ADMIN_PASSWORD_FILE=/run/secrets/confsync_admin
# Comptes locaux (false exige OIDC)
# AUTH_LOCAL=true
# ---------- OIDC (optionnel ; actif dès que OIDC_ISSUER est renseigné) ----------
# OIDC_ISSUER=https://auth.example.org/application/o/confsync/
# OIDC_CLIENT_ID=confsync
# OIDC_CLIENT_SECRET=
# OIDC_CLIENT_SECRET_FILE=/run/secrets/oidc_client
# URL complète déclarée chez l'IdP ; son chemin devient la route de callback
# OIDC_REDIRECT_URL=https://confsync.example.org/auth/callback
# OIDC_SCOPES=openid profile email # ajouter le scope qui fournit les groupes (ex. groups)
# OIDC_GROUPS_CLAIM=groups
# OIDC_ADMIN_GROUP=confsync-admins
# OIDC_OPERATOR_GROUP=confsync-operators
# OIDC_DEFAULT_ROLE=viewer # admin, operator ou viewer
Dockerfile
+2 -4
View File
@@ -16,14 +16,12 @@ RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w -X main.version=${VERSION}"
FROM alpine:3.24.2
RUN apk add --no-cache ca-certificates \
&& adduser -D -H -u 10001 confsync \
&& mkdir /data /config && chown confsync /data
&& mkdir /data && chown confsync /data
COPY --from=build /out/pve-confsync /usr/local/bin/pve-confsync
USER confsync
WORKDIR /data
VOLUME /data
ENV CONFSYNC_CONFIG=/config/confsync.yaml DATA_DIR=/data LISTEN=:8080
ENV DATA_DIR=/data LISTEN=:8080
EXPOSE 8080
HEALTHCHECK --interval=30s --timeout=3s CMD wget -qO- http://127.0.0.1:8080/healthz || exit 1
ENTRYPOINT ["/usr/local/bin/pve-confsync"]
# interface web ; la CLI reste disponible : docker compose run --rm confsync diff
CMD ["serve"]
README.md
+135 -105
View File
@@ -1,27 +1,69 @@
# pve-confsync
Synchronisation d'une partie de la configuration d'un cluster Proxmox VE 9.x
« référence » vers N clusters « clients », via l'API PVE et des tokens API.
Sens unique référence → clients. Pas de VM, pas de disques.
Réplication d'une partie de la configuration d'un cluster Proxmox VE 9.x **primaire**
vers une ou plusieurs **cibles**. Sens unique primaire → cibles. Pas de VM, pas de disques.
Tout se fait depuis l'interface web : plus de CLI ni de fichier YAML.
**État : phases 0 à 2.** CLI (export, diff et apply en YAML) et interface web
(écarts par client, dry-run et synchronisation par objet, par client ou global, journal,
audit) pour les options datacenter et le SDN (zones VLAN/VXLAN, VNets, subnets).
Testé contre un faux serveur PVE ; **pas encore testé sur les clusters de labo.**
**État : phases 0 à 2, version « full web ».** Ajout des clusters (un par un ou en lot),
choix du primaire, éléments répliqués et cibles choisis dans des menus, écarts par cible,
dry-run et réplication par objet, par cible ou global, journal, audit.
Testé contre un faux serveur PVE (API + SSH) ; **pas encore testé sur un vrai cluster.**
## Périmètre
| Type (`kind`) | API | Politique par défaut |
| Type (`kind`) | Accès | Politique par défaut |
|---|---|---|
| `datacenter-options` | `/cluster/options` | `tag-style`, `registered-tags`, `user-tag-access`, `next-id` en `sync`, le reste ignoré ; `webauthn`, `u2f`, `http_proxy` jamais synchronisés |
| `sdn-zone` (vlan, vxlan) | `/cluster/sdn/zones` | `nodes`, `bridge`, `peers`, `fabric`, `dns`, `reversedns`, `dnszone` en `override` (mapping client) ; le reste en `sync` |
| `sdn-vnet` | `/cluster/sdn/vnets` | `sync` |
| `sdn-subnet` | `/cluster/sdn/vnets/{vnet}/subnets` | `sync` |
| `datacenter-options` | API `/cluster/options` | `tag-style`, `registered-tags`, `user-tag-access`, `next-id` en `sync`, le reste ignoré ; `webauthn`, `u2f`, `http_proxy` jamais répliqués |
| `pve-group` | API `/access/groups` | `sync` |
| `pve-role` | API `/access/roles` | `sync` (rôles intégrés ignorés) |
| `pve-user` | API `/access/users` | `sync`, comptes du royaume `pve` uniquement |
| `pve-password` | SSH `/etc/pve/priv/shadow.cfg` | `sync` (haché recopié tel quel, jamais affiché) |
| `sdn-zone` (vlan, vxlan) | API `/cluster/sdn/zones` | `nodes`, `bridge`, `peers`, `fabric`, `dns`, `reversedns`, `dnszone` en `override` (mapping de la cible) ; le reste en `sync` |
| `sdn-vnet` | API `/cluster/sdn/vnets` | `sync` |
| `sdn-subnet` | API `/cluster/sdn/vnets/{vnet}/subnets` | `sync` |
Hors périmètre v1 : EVPN (et les zones simple, QinQ, faucet, avec leurs VNets, ignorées
avec un avertissement), firewall, storage, pools/ACL, jobs, secrets.
Les comptes du realm pve et leurs mots de passe forment un seul élément à répliquer ;
ils exigent leurs groupes, que le menu « Éléments répliqués » coche avec eux. Un parent absent de la cible et non répliqué
est une erreur bloquante.
Détail des routes, schémas et droits, avec les sources : [docs/api-pve.md](docs/api-pve.md).
Hors périmètre : EVPN (et les zones simple, QinQ, faucet, avec leurs VNets, ignorées
avec un avertissement), firewall, storage, pools, ACL, jobs, comptes `@pam` et autres royaumes.
Détail des routes, schémas et accès, avec les sources : [docs/api-pve.md](docs/api-pve.md).
## Réplication de nœuds
Deuxième mode, séparé du premier dans l'interface (sélecteur « Clusters (API) | Nœuds
(fichiers) » en haut de page) : un **profil** réplique la configuration système d'un
**nœud maître** vers des **nœuds cibles**, du même cluster ou d'un autre.
| Élément | Accès | Rechargement | Particularités |
|---|---|---|---|
| `network` | API `/nodes/{node}/network` (écrit `interfaces.new`) | `PUT /nodes/{node}/network` (ifreload -a) | `cidr`, `cidr6`, `gateway`, `gateway6` en `override` : chaque nœud garde ses adresses ; mapping des noms d'interfaces par cible (eno1=enp65s0f0, VLAN suivis) ; interface physique absente = bloquant ; options que l'API n'écrit pas (bond-miimon, post-up…) signalées, non répliquées |
| `dns` | API `/nodes/{node}/dns` | aucun | domaine de recherche obligatoire pour l'API |
| `multipath` | SSH `/etc/multipath.conf` | `multipathd reconfigure` | |
| `wwids` | SSH `/etc/multipath/wwids` | `multipathd reconfigure` | ajout seul des WWID du maître |
| `chrony` | SSH `/etc/chrony/chrony.conf`, `sources.d/*.sources` | `systemctl restart chrony` | |
| `sysctl` | SSH `/etc/sysctl.d/*.conf` | `sysctl --system` | |
| `interfaces-d` | SSH `/etc/network/interfaces.d/*` | `ifreload -a` | `interfaces.d/sdn` (généré par le SDN) jamais touché |
Jamais répliqués : `/etc/pve`, corosync, hostname, machine-id, clés SSH, IQN iSCSI, `/etc/frr`.
- **Accès** : « Découvrir les nœuds » lit `/cluster/status` de chaque cluster (adresse de
chaque nœud). SSH en root sur chaque nœud, avec le mot de passe de son cluster ou le sien ;
clé d'hôte épinglée au premier test, qui vérifie aussi que le nœud répond bien sous son nom.
- **Règles** : politiques par champ (réseau, DNS), lignes protégées par expression régulière
pour les fichiers (gardées de la cible), valeurs propres à une cible (`vmbr1 cidr=…`).
- **Apply** (administrateurs), un nœud après l'autre, arrêt au premier échec : un script SSH
vérifie que chaque fichier est celui du dry-run (SHA-256), sauvegarde dans
`/var/lib/pve-confsync/backup/<date>/` avec un `restore.sh`, écrit de façon atomique
(fichier temporaire puis `mv`, droits et propriétaire gardés) ; l'API écrit les interfaces
et le DNS ; les rechargements suivent. Un rechargement en échec restaure les sauvegardes.
- **Filet réseau** : avant de recharger le réseau, un timer systemd est armé sur le nœud
(`pve-confsync-restore-<date>`) : sans nouvelle connexion SSH et API dans les 120 s, le nœud
relance `restore.sh` et `ifreload -a` de lui-même. Confirmation tapée (nom du nœud, ou du
profil en global) dès qu'un plan recharge le réseau. Refus si la cible a déjà des
changements réseau en attente (`interfaces.new`).
## Fonctionnement
@@ -29,85 +71,81 @@ Détail des routes, schémas et droits, avec les sources : [docs/api-pve.md](doc
lecture source -> normalisation -> diff -> plan -> apply -> vérification
```
- **Lecture** : la référence est lue en configuration SDN *appliquée* (`running=1`) ; les
clients en configuration courante. Chaque cluster est refusé si un nœud en ligne n'est
- **Lecture** : le primaire est lu en configuration SDN *appliquée* (`running=1`) ; les
cibles en configuration courante. Chaque cluster est refusé si un nœud en ligne n'est
pas en PVE 9.x (`/nodes/{node}/version`).
- **Normalisation** : objets JSON génériques → champs canoniques (chaînes, listes triées,
property strings à clés triées). Les champs calculés par PVE sont retirés.
- **Politiques** : `sync` (copie de la référence), `ignore`, `override` (valeur fournie
uniquement par le `mapping` du client). Le mapping d'un client est prioritaire sur la
référence. Résolution : politique du client → politique globale → défaut de l'adaptateur.
- **Politiques** : `sync` (copie du primaire), `ignore`, `override` (valeur fournie
uniquement par le `mapping` de la cible). Le mapping d'une cible est prioritaire sur le
primaire. Résolution : politique de la cible → politique globale → défaut de l'adaptateur.
- **Plan** : création / mise à jour (champs modifiés, champs à supprimer) ; suppression
d'objets uniquement avec `prune` (option explicite), dépendants d'abord. Erreurs bloquantes :
champ obligatoire absent à la création (ex. `bridge` d'une zone VLAN sans mapping),
champ immuable modifié (type de zone, vnet d'un subnet).
- **Apply SDN** : `POST /cluster/sdn/lock` → écritures avec `lock-token` → `PUT /cluster/sdn`
(commit + rechargement réseau, suivi de la tâche). Toute erreur avant le commit déclenche
`POST /cluster/sdn/rollback`. Le verrou est refusé si le client a déjà des changements SDN
`POST /cluster/sdn/rollback`. Le verrou est refusé si la cible a déjà des changements SDN
en attente : on ne mélange pas nos modifications avec celles de quelqu'un d'autre.
- **Vérification** : relecture du client, nouveau plan qui doit être vide, et aucune zone en
- **Apply comptes** : API pour groupes, rôles et comptes ; mots de passe par SSH (script
perl sous `lock_shadow_config`, comme `pveum passwd`).
- **Vérification** : relecture de la cible, nouveau plan qui doit être vide, et aucune zone en
`error` sur les nœuds (`/nodes/{node}/sdn/zones`).
## Utilisation
## Démarrage
```sh
go build -o pve-confsync ./cmd/pve-confsync
cp examples/confsync.example.yaml confsync.yaml # puis adapter
./pve-confsync check # accès, version 9.x, droits des tokens
./pve-confsync export -o ref.yaml # état normalisé de la référence (versionnable dans Git)
./pve-confsync diff # écarts par client ; code retour 2 s'il y en a
./pve-confsync diff -format json
./pve-confsync apply # dry-run (défaut) : affiche le plan, n'écrit rien
./pve-confsync apply -client site-a -confirm # applique ; SDN : saisir le nom du client
./pve-confsync apply -source ref.yaml -confirm -yes # depuis un export Git, sans question
go run ./cmd/pve-confsync -demo # démo : faux clusters PVE, http://localhost:8080
docker compose -f docker-compose.demo.yml up -d --build # même démo en conteneur
docker compose up -d # production : données dans le volume /data
```
Avec docker compose (images et utilisateur non privilégié alignés sur Logstream) :
La clé AES-256 des mots de passe en base est créée au premier démarrage dans
`/data/secret.key` (à sauvegarder : sans elle, les mots de passe enregistrés sont perdus).
En production, on peut la garder hors du volume en secret docker (`CONFSYNC_SECRET_KEY_FILE`,
voir les commentaires de `docker-compose.yml`). Port publié : `CONFSYNC_PORT` (`.env`, 8080 par défaut).
```sh
mkdir -p secrets && printf '%s' 'xxxxxxxx-...' > secrets/pve_ref
chmod 600 secrets/* && sudo chown 10001 secrets/* # lisibles par l'utilisateur du conteneur uniquement
docker compose run --rm confsync check
docker compose run --rm confsync apply -confirm
```
## Interface web
```sh
go run ./cmd/pve-confsync serve -demo # démo : faux clusters PVE, http://localhost:8080
docker compose up -d confsync-web # production : données dans le volume /data
```
Le mode démo démarre une référence PVE 9.1, deux clients avec des écarts (dont un mapping
Le mode démo démarre un primaire PVE 9.1, deux cibles avec des écarts (dont un mapping
manquant, bloquant) et un cluster PVE 8 refusé. Comptes `admin`, `operateur` et `lecteur`,
mot de passe `confsync-demo`.
- **Écarts** : une carte par cluster (version, compteurs à créer / modifier / erreurs /
hors référence), puis les objets du client choisi ; le détail d'un objet montre pour
chaque champ la politique, la valeur de référence, le mapping, la valeur actuelle et
la valeur après synchro.
- **Dry-run / Synchroniser** par objet (avec ses dépendances : zone et vnet parents à
créer), par client ou global. La synchronisation passe toujours par un dry-run : le
serveur recalcule le plan et refuse s'il a changé depuis (empreinte du plan). Avec des
changements SDN, il faut taper le nom du client (celui de la référence en global).
Une seule synchronisation à la fois, suivie étape par étape.
- **Journal** des dry-runs et synchronisations (plans, étapes, résultats) et **Audit**
(connexions, actions, modifications des paramètres ; admin).
- **Paramètres** : clusters (test d'accès et des droits, TLS, mapping et politiques par
client), types synchronisés, sélection, politiques, suppression (désactivée par défaut),
utilisateurs locaux, import / export YAML (format de la CLI), compte, apparence.
**Base vide** : la page de connexion propose de créer le premier administrateur
(12 caractères minimum). Tant que ce n'est pas fait, **le premier qui atteint l'instance
devient administrateur** : ne pas l'exposer avant. Alternative : `CONFSYNC_ADMIN_USER` et
`CONFSYNC_ADMIN_PASSWORD` / `_FILE` créent ce compte au démarrage.
Rôles : `admin` (tout), `operator` (dry-run et synchronisation), `viewer` (lecture).
Les comptes locaux restent disponibles même avec OIDC.
## Interface web
- **Clusters** : ajout d'un cluster (URL `https://…:8006`, mot de passe root) ou d'une
liste (« nom URL » par ligne, même mot de passe), test (API, version 9.x, SSH), rôle
primaire / cible / désactivé. Un seul primaire, jamais écrit.
- **Éléments répliqués** et **Cibles** : menus de la vue Écarts ; les cibles décochées
sont exclues du global (choix mémorisé dans le navigateur).
- **Écarts** : une carte par cluster (version, compteurs à créer / modifier / erreurs /
hors primaire), puis les objets de la cible choisie ; le détail d'un objet montre pour
chaque champ la politique, la valeur du primaire, le mapping, la valeur actuelle et
la valeur après réplication. Les hachés sont masqués (`<masqué xxxxxx>`).
- **Dry-run / Répliquer** par objet (avec ses parents à créer), par cible ou global.
Dans le dry-run, chaque élément peut être exclu de l'exécution (« Ne pas répliquer ») ;
le dry-run est alors relancé sans lui, sauf ce dont les autres ont besoin.
La réplication passe toujours par un dry-run : le serveur recalcule le plan et refuse
s'il a changé depuis (empreinte du plan). Avec des changements SDN, il faut taper le
nom de la cible (celui du primaire en global). Une seule réplication à la fois.
- **Journal** des dry-runs et réplications et **Audit** (connexions, actions,
modifications ; admin).
- **Paramètres** : sélection, politiques, mapping par cible, suppression (désactivée par
défaut), utilisateurs locaux, compte, apparence.
Rôles : `admin` (tout), `operator` (dry-run et réplication), `viewer` (lecture).
Les comptes locaux restent disponibles même avec OIDC. Toutes les variables sont reprises,
commentées, dans [`.env.example`](.env.example) (`cp .env.example .env`).
| Variable | Défaut | Rôle |
|---|---|---|
| `LISTEN` | `:8080` | adresse d'écoute |
| `DATA_DIR` | `/data` | base SQLite, `secret.key`, `session.key` |
| `CONFSYNC_SECRET_KEY` / `_FILE` | `DATA_DIR/secret.key` (créée) | clé AES-256 (hex ou base64) des secrets en base |
| `CONFSYNC_ADMIN_USER`, `CONFSYNC_ADMIN_PASSWORD` / `_FILE` | `admin`, aléatoire (journalisé une fois) | premier administrateur, base vide |
| `CONFSYNC_IMPORT` | | YAML de la CLI importé si la base n'a aucun cluster |
| `CONFSYNC_SECRET_KEY` / `_FILE` | `DATA_DIR/secret.key` (créée) | clé AES-256 (hex ou base64) des mots de passe en base |
| `CONFSYNC_ADMIN_USER`, `CONFSYNC_ADMIN_PASSWORD` / `_FILE` | `admin`, vide | premier administrateur sans passer par la page web |
| `AUTH_LOCAL` | `true` | comptes locaux |
| `OIDC_ISSUER`, `OIDC_CLIENT_ID`, `OIDC_CLIENT_SECRET` / `_FILE`, `OIDC_REDIRECT_URL` | | OIDC (code + PKCE) ; `OIDC_REDIRECT_URL` complète, par ex. `https://confsync.example.org/auth/callback` |
| `OIDC_SCOPES`, `OIDC_GROUPS_CLAIM` | `openid profile email`, `groups` | ajouter le scope qui fournit les groupes chez l'IdP |
@@ -117,68 +155,60 @@ Les comptes locaux restent disponibles même avec OIDC.
## Sécurité
- Secrets des tokens : jamais dans le YAML (`token_secret` en clair est refusé) ; lus depuis
une variable d'environnement (`token_secret_env`) ou un fichier (`token_secret_file`,
secret docker). Ils ne sont ni affichés ni journalisés. Dans l'interface, ils sont chiffrés en base
(AES-256-GCM, clé hors base) et jamais renvoyés au navigateur ; l'export YAML les remplace
par `token_secret_env`.
- **Accès PVE en `root@pam`** (API par ticket, SSH avec le même mot de passe) : nécessaire
pour les mots de passe du royaume `pve`, absents de l'API. Le compte root ne doit pas
avoir de TFA (refusé). Le primaire n'est jamais écrit.
- **Épinglage au premier test** : empreinte SHA-256 du certificat TLS (mode par défaut,
adapté aux certificats auto-signés PVE) et clé d'hôte SSH. Une clé SSH différente
ensuite bloque l'accès ; l'accepter demande une action explicite (« faire confiance à la
nouvelle clé »). Changer l'URL ou le nœud SSH oublie les empreintes. Autres modes TLS :
CA (PEM), CA système, `insecure` (labo).
- **Mots de passe root** chiffrés en base (AES-256-GCM, clé hors base), jamais renvoyés au
navigateur ni journalisés. Les hachés du shadow sont masqués dans l'UI, le journal et
les réponses JSON.
- Interface : mots de passe PBKDF2-SHA256 (600 000 itérations), cookie de session signé
(HttpOnly, SameSite=Lax, Secure derrière HTTPS) invalidé au changement de mot de passe,
en-tête `X-Confsync` exigé sur les écritures (CSRF), CSP sans script ni style en ligne.
- TLS : `fingerprint` (pinning SHA-256 du certificat, adapté aux certificats auto-signés PVE),
`ca_file`, ou `insecure: true` réservé au labo.
- Tokens (privilege separation activée, l'ACL doit être posée sur l'utilisateur et le token) :
```sh
# référence : lecture seule
pveum role add ConfSyncRO --privs "Sys.Audit SDN.Audit"
pveum user add confsync@pve
pveum user token add confsync@pve ro --privsep 1
pveum acl modify / --users confsync@pve --roles ConfSyncRO
pveum acl modify / --tokens 'confsync@pve!ro' --roles ConfSyncRO
# clients : Sys.Modify uniquement si les options datacenter sont synchronisées
pveum role add ConfSyncRW --privs "Sys.Audit Sys.Modify SDN.Audit SDN.Allocate"
```
`check` signale un token de référence capable d'écrire et un token client à qui il manque
un droit.
## Arborescence
```
cmd/pve-confsync/ CLI (check, export, diff, apply, serve)
cmd/pve-confsync/ serveur web (seul point d'entrée)
internal/service/ analyse pour l'UI : lecture parallèle, plans, lignes de diff par champ
internal/store/ SQLite : clusters (secrets chiffrés), paramètres, utilisateurs, journal, audit
internal/store/ SQLite : clusters (mots de passe chiffrés, rôles, empreintes), utilisateurs, journal, audit
internal/server/ HTTP : auth locale + OIDC, rôles, API, jobs de synchronisation
internal/demo/ faux clusters du mode démo
web/ UI embarquée (index.html, app.js, style.css, login.html)
internal/pve/ client API : token, TLS, form-encoding, version, tâches
internal/pve/pvetest/ faux serveur PVE 9.x pour les tests
internal/model/ objets génériques, snapshot YAML, valeurs canoniques
internal/pve/ client API : ticket root@pam, TLS épinglé, form-encoding, version, tâches
internal/sshx/ SSH root, clé d'hôte épinglée
internal/pve/pvetest/ faux serveur PVE 9.x (API + SSH) pour les tests
internal/model/ objets génériques, valeurs canoniques
internal/adapter/ un adaptateur par type : routes, champs calculés, locaux, immuables
internal/config/ configuration YAML : clusters, politiques, mappings, sélection
internal/config/ configuration en mémoire : clusters, politiques, mappings, sélection
internal/engine/ lecture, plan, apply (verrou SDN, rollback), vérification
internal/nodesync/ réplication de nœuds : réseau et DNS (API), fichiers (SSH), diff, sauvegarde, restauration
docs/api-pve.md routes et schémas vérifiés dans les sources PVE
```
Tests : `go test ./...` (bout en bout contre le faux serveur : synchro complète, mapping
manquant, rollback sur erreur, refus de PVE 8, prune et ordre de suppression, forme
canonique des options datacenter, CLI dry-run/apply/export ; sélection d'objets et dépendances ; chiffrement, utilisateurs ;
serveur de bout en bout : rôles, CSRF, dry-run, empreinte périmée, confirmation SDN, apply).
Tests : `go test ./...` (bout en bout contre le faux serveur : réplication SDN et comptes
avec mots de passe par SSH, mapping manquant, parent absent, rollback sur erreur, refus de
PVE 8, prune et ordre de suppression ; sélection et dépendances ; chiffrement, migration
de schéma, premier administrateur ; serveur depuis une base vide : création de l'admin,
ajout des clusters, épinglage, primaire, réplication globale, primaire jamais écrit).
## Phasage
0. **CLI** export / diff / apply YAML (options datacenter, tags, SDN).
1. UI lecture seule : clusters, diff visuel par client. Ergonomie reprise de Logstream
0. Moteur export / diff / apply (options datacenter, tags, SDN) ; la CLI a été retirée.
1. UI lecture seule : clusters, diff visuel par cible. Ergonomie reprise de Logstream
(`web/index.html`, `style.css`, `app.js`, `login.html`, auth locale + OIDC, audit) ;
SQLite (clusters, profils, mappings, historique, audit), secrets chiffrés.
2. UI d'action : Dry-run / Synchroniser (par objet, par client, global), journal des exécutions. ← ici
2. UI d'action : Dry-run / Répliquer (par objet, par cible, global), journal ; gestion
des clusters en web, primaire / cibles, comptes et mots de passe du royaume pve. ← ici
3. Planification, détection de dérive en lecture seule, notifications.
## Reste à faire
- Valider sur les deux clusters de labo : forme réelle des réponses (`nodes`, `dhcp-range`,
- Valider sur un vrai cluster : écriture du shadow par SSH, forme de `/access/users?full=1`, forme réelle des réponses (`nodes`, `dhcp-range`,
booléens), comportement de `PUT /cluster/options` avec `delete`.
- Exploiter `GET /cluster/sdn/dry-run` (diff frr / interfaces, PVE ≥ 9.2 environ) pour
enrichir le dry-run SDN.
cmd/pve-confsync/main.go
+135 -349
View File
@@ -1,88 +1,39 @@
// pve-confsync synchronizes part of the configuration of a reference Proxmox VE
// cluster (datacenter options, SDN VLAN/VXLAN zones, vnets, subnets) to client
// clusters through the PVE API: command line (check, export, diff, apply) and
// web UI (serve).
// pve-confsync replicates part of the configuration of a primary Proxmox VE 9.x
// cluster (datacenter options, accounts of the pve realm with their passwords,
// groups, roles, SDN VLAN/VXLAN zones, vnets, subnets) to target clusters.
// Everything is done from the web UI: this program only starts the server.
// Settings come from the environment, like Logstream (see README).
package main
import (
"bufio"
"context"
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"io/fs"
"log"
"net/http"
"os"
"os/signal"
"sort"
"path/filepath"
"strconv"
"strings"
"syscall"
"time"
"gopkg.in/yaml.v3"
"pve-confsync/internal/adapter"
"pve-confsync/internal/config"
"pve-confsync/internal/engine"
"pve-confsync/internal/model"
"pve-confsync/internal/pve"
"pve-confsync/internal/service"
"pve-confsync/internal/demo"
"pve-confsync/internal/server"
"pve-confsync/internal/store"
"pve-confsync/web"
)
var version = "dev"
const usage = `pve-confsync %s - synchronisation de configuration Proxmox VE 9.x (référence -> clients)
Usage : pve-confsync [-config fichier] <commande> [options]
Commandes :
check vérifie l'accès, la version PVE (9.x) et les droits des tokens
export exporte l'état normalisé d'un cluster en YAML (défaut : la référence)
diff affiche les écarts référence -> clients (code retour 2 s'il y en a)
apply dry-run par défaut ; applique avec -confirm
serve interface web (clusters, diff, dry-run, synchronisation, journal)
version affiche la version
Lancer "pve-confsync <commande> -h" pour les options d'une commande.
`
func main() {
global := flag.NewFlagSet("pve-confsync", flag.ExitOnError)
cfgFile := global.String("config", envOr("CONFSYNC_CONFIG", "confsync.yaml"), "fichier de configuration")
global.Usage = func() { fmt.Fprintf(os.Stderr, usage, version); global.PrintDefaults() }
_ = global.Parse(os.Args[1:])
args := global.Args()
if len(args) == 0 {
global.Usage()
os.Exit(1)
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
var err error
code := 0
switch args[0] {
case "version":
fmt.Println(version)
return
case "check":
err = cmdCheck(ctx, *cfgFile, args[1:])
case "export":
err = cmdExport(ctx, *cfgFile, args[1:])
case "diff":
code, err = cmdDiff(ctx, *cfgFile, args[1:])
case "apply":
err = cmdApply(ctx, *cfgFile, args[1:])
case "serve":
err = cmdServe(ctx, args[1:])
default:
global.Usage()
os.Exit(1)
if err := serve(ctx, os.Args[1:]); err != nil {
log.Fatal(err)
}
if err != nil {
fmt.Fprintln(os.Stderr, "erreur :", err)
os.Exit(1)
}
os.Exit(code)
}
func envOr(k, def string) string {
@@ -92,318 +43,153 @@ func envOr(k, def string) string {
return def
}
// ---- check ----
// serve starts the web UI.
func serve(ctx context.Context, args []string) error {
fl := flag.NewFlagSet("pve-confsync", flag.ExitOnError)
listen := fl.String("listen", envOr("LISTEN", ":8080"), "adresse d'écoute HTTP")
dataDir := fl.String("data", envOr("DATA_DIR", "/data"), "répertoire des données (base SQLite, clés)")
demoMode := fl.Bool("demo", envBool("CONFSYNC_DEMO", false), "mode démo : faux clusters PVE en mémoire, base temporaire")
webDir := fl.String("web", os.Getenv("WEB_DIR"), "servir les pages depuis ce répertoire au lieu de celles embarquées (développement)")
_ = fl.Parse(args)
func cmdCheck(ctx context.Context, cfgFile string, args []string) error {
fs := flag.NewFlagSet("check", flag.ExitOnError)
_ = fs.Parse(args)
cfg, err := config.Load(cfgFile)
if *demoMode && os.Getenv("DATA_DIR") == "" && !flagSet(fl, "data") {
dir, err := os.MkdirTemp("", "confsync-demo-")
if err != nil {
return err
}
var errs []error
check := func(cl config.Cluster, role string, wantWrite bool) {
c, err := cl.PVE()
if err != nil {
errs = append(errs, err)
return
defer os.RemoveAll(dir)
*dataDir = dir
}
vs, err := c.CheckVersion(ctx)
for _, v := range vs {
fmt.Printf("%-10s %-16s nœud %-12s PVE %s\n", role, cl.Name, v.Node, v.Version)
if err := os.MkdirAll(*dataDir, 0o700); err != nil {
return err
}
if err != nil {
errs = append(errs, err)
return
}
perms, err := c.Permissions(ctx)
if err != nil {
fmt.Printf(" avertissement : droits du token non lisibles (%v)\n", err)
return
}
for _, w := range service.PrivilegeWarnings(perms, wantWrite, cfg.HasKind(model.KindDatacenter)) {
fmt.Println(" avertissement :", w)
}
}
check(cfg.Reference, "référence", false)
for _, cl := range cfg.Clients {
check(cl.Cluster, "client", true)
}
return errors.Join(errs...)
}
// ---- export ----
func cmdExport(ctx context.Context, cfgFile string, args []string) error {
fs := flag.NewFlagSet("export", flag.ExitOnError)
name := fs.String("cluster", "", "cluster à exporter (défaut : la référence)")
out := fs.String("o", "-", "fichier de sortie YAML")
_ = fs.Parse(args)
cfg, err := config.Load(cfgFile)
key, keySrc, err := store.LoadKey(*dataDir)
if err != nil {
return err
}
cl, mode := cfg.Reference, adapter.ReadRunning
if *name != "" && *name != cfg.Reference.Name {
c, ok := cfg.Client(*name)
if !ok {
return fmt.Errorf("cluster %q inconnu", *name)
}
cl, mode = c.Cluster, adapter.ReadCurrent
}
c, err := cl.PVE()
st, err := store.Open(filepath.Join(*dataDir, "confsync.db"), key)
if err != nil {
return err
}
snap, err := engine.Read(ctx, c, cfg.Kinds, mode)
defer st.Close()
log.Printf("database %s, encryption key from %s", filepath.Join(*dataDir, "confsync.db"), keySrc)
if *demoMode {
clusters, err := demo.Start(ctx, st)
if err != nil {
return fmt.Errorf("demo: %w", err)
}
defer clusters.Close()
log.Printf("DEMO mode: fake PVE clusters started; accounts admin, operateur, lecteur, password %q", demo.Password)
} else if err := bootstrap(ctx, st); err != nil {
return err
} else if n, _ := st.CountUsers(ctx); n == 0 {
log.Printf("empty database: open the web UI to create the first administrator")
}
ttl, err := time.ParseDuration(envOr("SESSION_TTL", "12h"))
if err != nil {
return fmt.Errorf("SESSION_TTL: %w", err)
}
var static fs.FS = web.FS
if *webDir != "" {
static = os.DirFS(*webDir)
}
srv, err := server.New(server.Options{
Store: st,
Static: static,
Version: version,
Demo: *demoMode,
Auth: server.AuthConfig{
Local: envBool("AUTH_LOCAL", true),
Issuer: os.Getenv("OIDC_ISSUER"),
ClientID: os.Getenv("OIDC_CLIENT_ID"),
ClientSecret: secretEnv("OIDC_CLIENT_SECRET"),
RedirectURL: os.Getenv("OIDC_REDIRECT_URL"),
Scopes: os.Getenv("OIDC_SCOPES"),
GroupsClaim: os.Getenv("OIDC_GROUPS_CLAIM"),
AdminGroup: os.Getenv("OIDC_ADMIN_GROUP"),
OperatorGroup: os.Getenv("OIDC_OPERATOR_GROUP"),
DefaultRole: os.Getenv("OIDC_DEFAULT_ROLE"),
SessionTTL: ttl,
DataDir: *dataDir,
Demo: *demoMode,
},
})
if err != nil {
return err
}
warn(snap.Warnings)
return writeYAML(*out, snap)
}
func writeYAML(file string, v any) error {
var w io.Writer = os.Stdout
if file != "-" {
f, err := os.Create(file)
if err != nil {
hs := &http.Server{Addr: *listen, Handler: srv.Handler(), ReadHeaderTimeout: 10 * time.Second}
errc := make(chan error, 1)
go func() { errc <- hs.ListenAndServe() }()
log.Printf("pve-confsync %s: web UI on %s", version, *listen)
select {
case err := <-errc:
return err
case <-ctx.Done():
}
defer f.Close()
w = f
}
enc := yaml.NewEncoder(w)
enc.SetIndent(2)
if err := enc.Encode(v); err != nil {
return err
}
return enc.Close()
log.Printf("stopping: waiting for the synchronizations in progress")
sctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = hs.Shutdown(sctx)
srv.Wait()
return nil
}
func loadSnapshot(file string) (model.Snapshot, error) {
var s model.Snapshot
raw, err := os.ReadFile(file)
if err != nil {
return s, err
// bootstrap: the database starts empty, the first administrator is created
// from the login page. CONFSYNC_ADMIN_USER / CONFSYNC_ADMIN_PASSWORD (optional)
// create it at startup instead, for an unattended deployment.
func bootstrap(ctx context.Context, st *store.Store) error {
pass := secretEnv("CONFSYNC_ADMIN_PASSWORD")
if pass == "" {
return nil
}
if err := yaml.Unmarshal(raw, &s); err != nil {
return s, fmt.Errorf("%s: %w", file, err)
user := envOr("CONFSYNC_ADMIN_USER", "admin")
err := st.CreateFirstAdmin(ctx, user, pass)
switch {
case err == nil:
log.Printf("first administrator %q created from the environment", user)
case err != store.ErrSetupDone:
return fmt.Errorf("first administrator: %w", err)
}
for i := range s.Objects {
s.Objects[i].Fields = model.Normalize(s.Objects[i].Fields)
if a, err := adapter.ByKind(s.Objects[i].Kind); err == nil {
a.Canonicalize(s.Objects[i].Fields)
} else {
return s, fmt.Errorf("%s: %w", file, err)
}
}
return s, nil
return nil
}
// ---- diff / apply ----
type target struct {
cl *config.Client
client *pve.Client
plan engine.Plan
// secretEnv reads NAME, or the file named by NAME_FILE (docker secret).
func secretEnv(name string) string {
if v := os.Getenv(name); v != "" {
return v
}
if f := os.Getenv(name + "_FILE"); f != "" {
b, err := os.ReadFile(f)
if err != nil {
log.Printf("%s_FILE: %v", name, err)
return ""
}
return strings.TrimSpace(string(b))
}
return ""
}
// prepare reads the source (live reference or YAML snapshot) and each selected
// client, and computes the plans.
func prepare(ctx context.Context, cfg *config.Config, source, only string) ([]target, error) {
var ref model.Snapshot
var err error
if source != "" {
ref, err = loadSnapshot(source)
} else {
var c *pve.Client
if c, err = cfg.Reference.PVE(); err == nil {
ref, err = engine.Read(ctx, c, cfg.Kinds, adapter.ReadRunning)
}
func envBool(name string, def bool) bool {
v := os.Getenv(name)
if v == "" {
return def
}
b, err := strconv.ParseBool(v)
if err != nil {
return nil, err
log.Printf("%s=%q: true or false expected, using %t", name, v, def)
return def
}
warn(ref.Warnings)
var ts []target
for i := range cfg.Clients {
cl := &cfg.Clients[i]
if only != "" && cl.Name != only {
continue
}
c, err := cl.PVE()
if err != nil {
return nil, err
}
cur, err := engine.Read(ctx, c, cfg.Kinds, adapter.ReadCurrent)
if err != nil {
return nil, err
}
p := engine.MakePlan(cfg, cl, ref.Objects, cur.Objects)
p.Warnings = append(p.Warnings, cur.Warnings...)
ts = append(ts, target{cl: cl, client: c, plan: p})
}
if only != "" && len(ts) == 0 {
return nil, fmt.Errorf("client %q inconnu", only)
}
return ts, nil
return b
}
func cmdDiff(ctx context.Context, cfgFile string, args []string) (int, error) {
fs := flag.NewFlagSet("diff", flag.ExitOnError)
only := fs.String("client", "", "limiter à un client")
source := fs.String("source", "", "lire la référence depuis un export YAML au lieu de l'API")
format := fs.String("format", "text", "text ou json")
_ = fs.Parse(args)
cfg, err := config.Load(cfgFile)
if err != nil {
return 1, err
}
ts, err := prepare(ctx, cfg, *source, *only)
if err != nil {
return 1, err
}
code := 0
var plans []engine.Plan
for _, t := range ts {
plans = append(plans, t.plan)
if !t.plan.Empty() || len(t.plan.Errors) > 0 {
code = 2
}
}
if *format == "json" {
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
return code, enc.Encode(plans)
}
for _, p := range plans {
printPlan(os.Stdout, p)
}
return code, nil
}
func cmdApply(ctx context.Context, cfgFile string, args []string) error {
fs := flag.NewFlagSet("apply", flag.ExitOnError)
only := fs.String("client", "", "limiter à un client")
source := fs.String("source", "", "lire la référence depuis un export YAML au lieu de l'API")
confirm := fs.Bool("confirm", false, "appliquer réellement (sans cette option : dry-run)")
yes := fs.Bool("yes", false, "ne pas demander la confirmation interactive des changements SDN")
prune := fs.Bool("prune", false, "supprimer chez les clients les objets absents de la référence")
_ = fs.Parse(args)
cfg, err := config.Load(cfgFile)
if err != nil {
return err
}
cfg.Prune = cfg.Prune || *prune
ts, err := prepare(ctx, cfg, *source, *only)
if err != nil {
return err
}
var src model.Snapshot
if *source != "" {
src, _ = loadSnapshot(*source)
}
var errs []error
for _, t := range ts {
printPlan(os.Stdout, t.plan)
if t.plan.Empty() || len(t.plan.Errors) > 0 {
if len(t.plan.Errors) > 0 {
errs = append(errs, fmt.Errorf("client %s : plan en erreur, rien appliqué", t.cl.Name))
}
continue
}
if !*confirm {
fmt.Printf("[%s] dry-run : rien n'a été appliqué (ajouter -confirm)\n\n", t.cl.Name)
continue
}
if t.plan.HasSDN() && !*yes && !askSDN(t.cl.Name) {
fmt.Printf("[%s] changements SDN non confirmés, client ignoré\n\n", t.cl.Name)
continue
}
res := engine.Apply(ctx, t.client, t.plan)
for _, s := range res.Steps {
status := "ok"
if s.Error != "" {
status = "ERREUR " + s.Error
}
fmt.Printf("[%s] %s %s : %s\n", t.cl.Name, s.At.Format("15:04:05"), s.Action, status)
}
if res.Err != nil {
errs = append(errs, fmt.Errorf("client %s : %w", t.cl.Name, res.Err))
continue
}
replan := func(actual []model.Object) engine.Plan {
ref := src.Objects
if *source == "" {
// the reference is re-read so that verification uses the same input
c, err := cfg.Reference.PVE()
var r model.Snapshot
if err == nil {
r, err = engine.Read(ctx, c, cfg.Kinds, adapter.ReadRunning)
}
if err != nil {
return engine.Plan{Errors: []string{err.Error()}}
}
ref = r.Objects
}
return engine.MakePlan(cfg, t.cl, ref, actual)
}
if err := engine.Verify(ctx, t.client, replan, cfg.Kinds); err != nil {
errs = append(errs, fmt.Errorf("client %s : vérification : %w", t.cl.Name, err))
continue
}
fmt.Printf("[%s] appliqué et vérifié\n\n", t.cl.Name)
}
return errors.Join(errs...)
}
// askSDN requires typing the client name: an SDN apply reloads the network of every node.
func askSDN(client string) bool {
fi, err := os.Stdin.Stat()
if err != nil || fi.Mode()&os.ModeCharDevice == 0 {
fmt.Fprintln(os.Stderr, "entrée non interactive : utiliser -yes pour confirmer les changements SDN")
return false
}
fmt.Printf("Appliquer les changements SDN sur %q (rechargement réseau de tous les nœuds) ?\nTaper le nom du client pour confirmer : ", client)
line, _ := bufio.NewReader(os.Stdin).ReadString('\n')
return strings.TrimSpace(line) == client
}
func printPlan(w io.Writer, p engine.Plan) {
for _, m := range p.Warnings {
fmt.Fprintf(w, "[%s] avertissement : %s\n", p.Client, m)
}
for _, e := range p.Errors {
fmt.Fprintf(w, "[%s] ERREUR : %s\n", p.Client, e)
}
if p.Empty() {
fmt.Fprintf(w, "[%s] aucun écart\n\n", p.Client)
return
}
for _, ch := range p.Changes {
fmt.Fprintf(w, "[%s] %-6s %s/%s\n", p.Client, ch.Op, ch.Kind, ch.ID)
keys := make([]string, 0, len(ch.Set))
for k := range ch.Set {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
if old, ok := ch.Old[k]; ok {
fmt.Fprintf(w, " ~ %s : %s -> %s\n", k, model.Format(old), model.Format(ch.Set[k]))
} else {
fmt.Fprintf(w, " + %s = %s\n", k, model.Format(ch.Set[k]))
}
}
for _, k := range ch.Unset {
fmt.Fprintf(w, " - %s (était %s)\n", k, model.Format(ch.Old[k]))
}
}
fmt.Fprintln(w)
}
func warn(ws []string) {
for _, w := range ws {
fmt.Fprintln(os.Stderr, "avertissement :", w)
func flagSet(fs *flag.FlagSet, name string) bool {
found := false
fs.Visit(func(f *flag.Flag) {
if f.Name == name {
found = true
}
})
return found
}
cmd/pve-confsync/main_test.go
-53
View File
@@ -1,53 +0,0 @@
package main
import (
"context"
"fmt"
"os"
"path/filepath"
"testing"
"pve-confsync/internal/pve/pvetest"
)
func TestCLIDiffExportApply(t *testing.T) {
ref := pvetest.New("9.1.4")
defer ref.Close()
cli := pvetest.New("9.1.4")
defer cli.Close()
ref.Options["registered-tags"] = "prod"
ref.Pending.Zones["vlanz"] = map[string]any{"type": "vlan", "bridge": "vmbr0"}
ref.Pending.VNets["v10"] = map[string]any{"zone": "vlanz", "tag": 10}
ref.Commit()
dir := t.TempDir()
t.Setenv("SECRET", "s3cr3t")
cfg := filepath.Join(dir, "c.yaml")
_ = os.WriteFile(cfg, []byte(fmt.Sprintf(`reference: {name: ref, url: %s, token_id: a@pve!ro, token_secret_env: SECRET, insecure: true}
clients:
- {name: c1, url: %s, token_id: a@pve!rw, token_secret_env: SECRET, insecure: true,
mapping: {sdn-zone: {vlanz: {bridge: vmbr1}}}}
`, ref.URL, cli.URL)), 0o600)
ctx := context.Background()
if code, err := cmdDiff(ctx, cfg, nil); err != nil || code != 2 {
t.Fatalf("diff: code=%d err=%v", code, err)
}
snap := filepath.Join(dir, "ref.yaml")
if err := cmdExport(ctx, cfg, []string{"-o", snap}); err != nil {
t.Fatal(err)
}
// dry-run by default: nothing written
if err := cmdApply(ctx, cfg, []string{"-source", snap}); err != nil || len(cli.Calls) != 0 {
t.Fatalf("dry-run wrote: %v %v", err, cli.Calls)
}
if err := cmdApply(ctx, cfg, []string{"-source", snap, "-confirm", "-yes"}); err != nil {
t.Fatal(err)
}
if cli.Running.VNets["v10"] == nil || cli.Options["registered-tags"] != "prod" {
t.Fatalf("not applied: %v %v", cli.Running.VNets, cli.Options)
}
if code, err := cmdDiff(ctx, cfg, nil); err != nil || code != 0 {
t.Fatalf("diff after apply: code=%d err=%v", code, err)
}
}
cmd/pve-confsync/serve.go
-199
View File
@@ -1,199 +0,0 @@
package main
import (
"context"
"flag"
"fmt"
"io/fs"
"log"
"net/http"
"os"
"path/filepath"
"strconv"
"strings"
"time"
"pve-confsync/internal/config"
"pve-confsync/internal/demo"
"pve-confsync/internal/server"
"pve-confsync/internal/store"
"pve-confsync/web"
)
// cmdServe starts the web UI (phases 1 and 2). Settings come from flags or from
// the environment, like Logstream (see README).
func cmdServe(ctx context.Context, args []string) error {
fl := flag.NewFlagSet("serve", flag.ExitOnError)
listen := fl.String("listen", envOr("LISTEN", ":8080"), "adresse d'écoute HTTP")
dataDir := fl.String("data", envOr("DATA_DIR", "/data"), "répertoire des données (base SQLite, clés)")
demoMode := fl.Bool("demo", envBool("CONFSYNC_DEMO", false), "mode démo : faux clusters PVE en mémoire, base temporaire")
webDir := fl.String("web", os.Getenv("WEB_DIR"), "servir les pages depuis ce répertoire au lieu de celles embarquées (développement)")
_ = fl.Parse(args)
if *demoMode && os.Getenv("DATA_DIR") == "" && !flagSet(fl, "data") {
dir, err := os.MkdirTemp("", "confsync-demo-")
if err != nil {
return err
}
defer os.RemoveAll(dir)
*dataDir = dir
}
if err := os.MkdirAll(*dataDir, 0o700); err != nil {
return err
}
key, keySrc, err := store.LoadKey(*dataDir)
if err != nil {
return err
}
st, err := store.Open(filepath.Join(*dataDir, "confsync.db"), key)
if err != nil {
return err
}
defer st.Close()
log.Printf("database %s, encryption key from %s", filepath.Join(*dataDir, "confsync.db"), keySrc)
if *demoMode {
clusters, err := demo.Start(ctx, st)
if err != nil {
return fmt.Errorf("demo: %w", err)
}
defer clusters.Close()
log.Printf("DEMO mode: fake PVE clusters started; accounts admin, operateur, lecteur, password %q", demo.Password)
} else if err := bootstrap(ctx, st); err != nil {
return err
}
ttl, err := time.ParseDuration(envOr("SESSION_TTL", "12h"))
if err != nil {
return fmt.Errorf("SESSION_TTL: %w", err)
}
var static fs.FS = web.FS
if *webDir != "" {
static = os.DirFS(*webDir)
}
srv, err := server.New(server.Options{
Store: st,
Static: static,
Version: version,
Demo: *demoMode,
Auth: server.AuthConfig{
Local: envBool("AUTH_LOCAL", true),
Issuer: os.Getenv("OIDC_ISSUER"),
ClientID: os.Getenv("OIDC_CLIENT_ID"),
ClientSecret: secretEnv("OIDC_CLIENT_SECRET"),
RedirectURL: os.Getenv("OIDC_REDIRECT_URL"),
Scopes: os.Getenv("OIDC_SCOPES"),
GroupsClaim: os.Getenv("OIDC_GROUPS_CLAIM"),
AdminGroup: os.Getenv("OIDC_ADMIN_GROUP"),
OperatorGroup: os.Getenv("OIDC_OPERATOR_GROUP"),
DefaultRole: os.Getenv("OIDC_DEFAULT_ROLE"),
SessionTTL: ttl,
DataDir: *dataDir,
Demo: *demoMode,
},
})
if err != nil {
return err
}
hs := &http.Server{Addr: *listen, Handler: srv.Handler(), ReadHeaderTimeout: 10 * time.Second}
errc := make(chan error, 1)
go func() { errc <- hs.ListenAndServe() }()
log.Printf("pve-confsync %s: web UI on %s", version, *listen)
select {
case err := <-errc:
return err
case <-ctx.Done():
}
log.Printf("stopping: waiting for the synchronizations in progress")
sctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
_ = hs.Shutdown(sctx)
srv.Wait()
return nil
}
// bootstrap creates the first administrator and imports a YAML configuration
// on an empty database.
func bootstrap(ctx context.Context, st *store.Store) error {
n, err := st.CountUsers(ctx)
if err != nil {
return err
}
if n == 0 {
user := envOr("CONFSYNC_ADMIN_USER", "admin")
pass := secretEnv("CONFSYNC_ADMIN_PASSWORD")
generated := pass == ""
if generated {
pass = store.RandomPassword()
}
if err := st.SaveUser(ctx, user, store.RoleAdmin, pass); err != nil {
return fmt.Errorf("first administrator: %w", err)
}
if generated {
log.Printf("first administrator %q created, password: %s (change it in Settings > Account)", user, pass)
} else {
log.Printf("first administrator %q created", user)
}
}
if file := os.Getenv("CONFSYNC_IMPORT"); file != "" {
cls, err := st.Clusters(ctx)
if err != nil {
return err
}
if len(cls) > 0 {
return nil // imported once: the database is the reference from now on
}
cfg, err := config.Load(file)
if err != nil {
return fmt.Errorf("CONFSYNC_IMPORT: %w", err)
}
warns, err := st.ImportConfig(ctx, cfg, false)
if err != nil {
return fmt.Errorf("CONFSYNC_IMPORT: %w", err)
}
for _, w := range warns {
log.Printf("import: %s", w)
}
log.Printf("configuration imported from %s", file)
}
return nil
}
// secretEnv reads NAME, or the file named by NAME_FILE (docker secret).
func secretEnv(name string) string {
if v := os.Getenv(name); v != "" {
return v
}
if f := os.Getenv(name + "_FILE"); f != "" {
b, err := os.ReadFile(f)
if err != nil {
log.Printf("%s_FILE: %v", name, err)
return ""
}
return strings.TrimSpace(string(b))
}
return ""
}
func envBool(name string, def bool) bool {
v := os.Getenv(name)
if v == "" {
return def
}
b, err := strconv.ParseBool(v)
if err != nil {
log.Printf("%s=%q: true or false expected, using %t", name, v, def)
return def
}
return b
}
func flagSet(fs *flag.FlagSet, name string) bool {
found := false
fs.Visit(func(f *flag.Flag) {
if f.Name == name {
found = true
}
})
return found
}
docker-compose.demo.yml
+17
View File
@@ -0,0 +1,17 @@
# Démo de l'interface web, sans cluster PVE ni secret à créer :
# docker compose -f docker-compose.demo.yml up -d --build
# http://<hôte>:8080 ; comptes admin, operateur, lecteur ; mot de passe confsync-demo
# Faux clusters PVE en mémoire (primaire 9.1, deux cibles avec écarts, un PVE 8 refusé).
# Base temporaire : tout est remis à zéro au redémarrage du conteneur.
services:
confsync-demo:
build: .
image: pve-confsync:dev
container_name: pve-confsync-demo
command: ["-demo"]
restart: unless-stopped
ports:
- "${CONFSYNC_PORT:-8080}:8080"
environment:
TZ: ${TZ:-Europe/Paris}
DATA_DIR: "" # vide : base temporaire dans /tmp, rien n'est conservé
docker-compose.yml
+19 -47
View File
@@ -1,25 +1,22 @@
# Interface web (phases 1-2) :
# docker compose up -d confsync-web # http://<hôte>:8080 ; mot de passe admin dans les logs au 1er démarrage
# CLI de la phase 0, lancée à la demande avec confsync.yaml :
# docker compose run --rm confsync check
# docker compose run --rm confsync diff
# docker compose run --rm confsync export -o /data/ref.yaml
# docker compose run --rm confsync apply # dry-run
# docker compose run --rm confsync apply -confirm # application (confirmation SDN interactive)
# Interface web :
# docker compose up -d # http://<hôte>:${CONFSYNC_PORT:-8080}
# Base vide : la page de connexion crée le premier administrateur (ne pas exposer
# l'instance avant), sauf si CONFSYNC_ADMIN_PASSWORD(_FILE) est fourni.
# Clusters, mots de passe root (chiffrés en base) et réglages : tout se fait dans l'UI.
services:
confsync-web:
confsync:
build: .
image: pve-confsync:dev
container_name: pve-confsync-web
command: ["serve"]
container_name: pve-confsync
restart: unless-stopped
ports:
- "8080:8080" # ou derrière Traefik (HTTPS : cookies Secure)
- "${CONFSYNC_PORT:-8080}:8080" # port publié (.env) ; ou derrière Traefik (HTTPS : cookies Secure)
environment:
TZ: ${TZ:-Europe/Paris}
CONFSYNC_SECRET_KEY_FILE: /run/secrets/confsync_key # clé des secrets en base, hors volume /data
# Clé AES des mots de passe en base : créée au premier démarrage dans /data/secret.key.
# Pour la garder hors du volume /data (recommandé en production) : secret docker ci-dessous.
# CONFSYNC_SECRET_KEY_FILE: /run/secrets/confsync_key
# CONFSYNC_ADMIN_PASSWORD_FILE: /run/secrets/confsync_admin
# CONFSYNC_IMPORT: /config/confsync.yaml # import initial de la config CLI (base vide)
# OIDC_ISSUER: https://auth.example.org/application/o/confsync/
# OIDC_CLIENT_ID: confsync
# OIDC_CLIENT_SECRET_FILE: /run/secrets/oidc_client
@@ -30,40 +27,15 @@ services:
- path: .env
required: false
volumes:
- confsync-data:/data # SQLite, session.key
secrets:
- confsync_key
- confsync-data:/data # SQLite, secret.key, session.key
# secrets:
# - confsync_key
confsync:
image: pve-confsync:dev
container_name: pve-confsync
profiles: ["cli"] # pas démarré par "up" ; "run" l'active
command: ["check"]
environment:
TZ: ${TZ:-Europe/Paris}
PVE_SITE_B_SECRET: ${PVE_SITE_B_SECRET:-} # secrets par variable (voir token_secret_env)
env_file:
- path: .env
required: false
volumes:
- ./confsync.yaml:/config/confsync.yaml:ro
- confsync-cli:/data # exports YAML
secrets:
- pve_ref
- pve_site_a
stdin_open: true
tty: true
# Fichiers chmod 600, propriétaire 10001, montés dans /run/secrets/<nom>.
# confsync_key : openssl rand -hex 32 > secrets/confsync_key
secrets:
confsync_key:
file: ./secrets/confsync_key
pve_ref:
file: ./secrets/pve_ref
pve_site_a:
file: ./secrets/pve_site_a
# Optionnel : clé fournie en secret docker (fichier chmod 600, propriétaire 10001).
# mkdir -p secrets && openssl rand -hex 32 > secrets/confsync_key
# secrets:
# confsync_key:
# file: ./secrets/confsync_key
volumes:
confsync-data:
confsync-cli:
docs/api-pve.md
+65 -9
View File
@@ -1,12 +1,18 @@
# API Proxmox VE 9.x utilisée par pve-confsync
Vérifiée le 2026-10-13 dans les sources Proxmox (miroirs GitHub officiels), faute d'accès
Vérifiée le 2026-10-13 (comptes : 2026-10-23) dans les sources Proxmox (miroirs GitHub officiels), faute d'accès
à l'API viewer (https://pve.proxmox.com/pve-docs/api-viewer/) depuis l'environnement de
développement. Versions lues : pve-manager 9.2.21, libpve-network-perl 1.6.7, pve-cluster 9.1.6.
**À recouper avec l'API viewer et les deux clusters de labo avant la phase 1.**
Toutes les routes sont relatives à `/api2/json`. Authentification : en-tête
`Authorization: PVEAPIToken=USER@REALM!TOKENID=SECRET`. Écritures en
Toutes les routes sont relatives à `/api2/json`. Authentification par ticket du compte
`root@pam` : `POST /access/ticket` (`username`, `password`) renvoie `ticket` et
`CSRFPreventionToken` ([AccessControl.pm L198](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/AccessControl.pm#L198)) ; le ticket
est envoyé dans le cookie `PVEAuthCookie`, le jeton CSRF dans l'en-tête
`CSRFPreventionToken` sur les écritures. Durée de vie du ticket : 2 h
([AccessControl.pm L58](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/AccessControl.pm#L58)) ; pve-confsync se reconnecte après 90 min
ou sur un 401. Un compte root avec TFA renvoie `NeedTFA`
([L167](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/AccessControl.pm#L167)) : refusé. Écritures en
`application/x-www-form-urlencoded` (tableaux = paramètre répété), suppression de champs
via le paramètre `delete=a,b`.
@@ -17,7 +23,6 @@ via le paramètre `delete=a,b`.
| GET | `/cluster/status` | nœuds en ligne | Sys.Audit sur / |
| GET | `/nodes/{node}/version` | `version` (9.2.21), `release` (9.2) : refus si majeur ≠ 9 | — |
| GET | `/version` | repli pour un nœud isolé ([API2.pm](https://github.com/proxmox/pve-manager/blob/9c3f5e5258f73f31e273493232a79d615fd9e072/PVE/API2.pm#L86)) | — |
| GET | `/access/permissions` | droits effectifs du token (`check`) ([AccessControl.pm](https://github.com/proxmox/pve-access-control/blob/master/src/PVE/API2/AccessControl.pm#L510)) | — |
| GET | `/nodes/{node}/tasks/{upid}/status` | suivi de la tâche de rechargement SDN | — |
## Options datacenter (`datacenter.cfg`)
@@ -77,9 +82,60 @@ Champs (schémas : [VlanPlugin.pm](https://github.com/proxmox/pve-network/blob/c
- subnet : `vnet`, `gateway`, `snat`, `dnszoneprefix`, `dhcp-range` (tableau), `dhcp-dns-server`
- calculés par PVE, ignorés : `digest`, `state`, `pending`, et pour les subnets `cidr`, `network`, `mask`, `zone`
## Droits minimaux
## Comptes du royaume pve (vérifié le 2026-10-23, pve-access-control dff26b45, pve-cluster 86d4bb9e)
- **Référence** (lecture seule) : rôle avec `Sys.Audit` + `SDN.Audit` sur `/`.
- **Clients** : `Sys.Audit` + `SDN.Audit` + `SDN.Allocate` sur `/` (ou `/sdn`) ;
`Sys.Modify` sur `/` uniquement si les options datacenter sont synchronisées
(`PUT /cluster/options` n'a pas de droit plus fin).
| Objet | Liste / création | Élément | Notes |
|---|---|---|---|
| groupe | `/access/groups` ([Group.pm L26](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/Group.pm#L26)) | `/access/groups/{groupid}` | champ `comment` ; `users` calculé, ignoré |
| rôle | `/access/roles` ([Role.pm L31](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/Role.pm#L31)) | `/access/roles/{roleid}` (PUT) | rôles `special` (intégrés) ignorés, non modifiables ([L146](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/Role.pm#L146)) ; `privs` trié |
| utilisateur | `/access/users?full=1` ([User.pm L156](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/API2/User.pm#L156)) | `/access/users/{userid}` | seuls les `@pve` ; `tokens`, `realm-type`, champs de verrou TFA ignorés ; `groups` liste triée ; `password` jamais envoyé par l'API ; `root@pam` jamais supprimé |
**Mots de passe.** L'API ne renvoie pas les hachés : ils sont dans
`/etc/pve/priv/shadow.cfg` (lignes `user:hash:`, utilisateur sans royaume,
[Auth/PVE.pm L13-L50](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/Auth/PVE.pm#L13)). Lecture et écriture par SSH en root :
- lecture : `cat /etc/pve/priv/shadow.cfg` ;
- écriture : script perl sous `PVE::Auth::PVE::lock_shadow_config`
([L51](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/Auth/PVE.pm#L51)) avec `cfs_read_file` / `cfs_write_file`, comme
`store_password` ([L95-L98](https://github.com/proxmox/pve-access-control/blob/dff26b458651b4e1141ff0a41a296d2904b16228/src/PVE/Auth/PVE.pm#L95)) : le haché est recopié tel quel, sans
le recalculer. `cfs_read_file` sans version relit toujours le fichier
([Cluster.pm](https://github.com/proxmox/pve-cluster/blob/86d4bb9efc0956311c01dbf59a38fa0583b60a19/src/PVE/Cluster.pm)).
**À valider sur un vrai cluster** : script d'écriture du shadow, forme réelle de
`/access/users?full=1`, mise à jour d'un utilisateur avec champs vidés.
## Réplication de nœuds (vérifié le 2026-10-23, pve-manager f643ce4b, pve-common master)
| Méthode | Chemin | Usage |
|---|---|---|
| GET | `/nodes/{node}/network` ([Network.pm](https://github.com/proxmox/pve-manager/blob/f643ce4bceb99e8b23094cd04075405281eb3f27/PVE/API2/Network.pm)) | interfaces, lues dans `interfaces.new` s'il existe ; `lo` retiré |
| POST | `/nodes/{node}/network` | création (`iface`, `type` + champs) dans `interfaces.new` |
| PUT | `/nodes/{node}/network/{iface}` | mise à jour (`type` obligatoire, `delete=`) |
| DELETE | `/nodes/{node}/network/{iface}` | suppression (prune) |
| PUT | `/nodes/{node}/network` | renomme `interfaces.new` puis `ifreload -a` dans une tâche (UPID) |
| DELETE | `/nodes/{node}/network` | annule les changements en attente (supprime `interfaces.new`) |
| GET / PUT | `/nodes/{node}/dns` ([Nodes.pm](https://github.com/proxmox/pve-manager/blob/f643ce4bceb99e8b23094cd04075405281eb3f27/PVE/API2/Nodes.pm)) | `search` (obligatoire au PUT), `dns1` à `dns3` ; le PUT réécrit tous les serveurs |
Points d'attention :
- champs écrits (`$confdesc`) : `type`, `comments`, `comments6`, `autostart`, `bridge_vlan_aware`,
`bridge_vids`, `bridge_ports`, `ovs_*`, `slaves`, `bond_mode`, `bond-primary`,
`bond_xmit_hash_policy`, `vlan-raw-device`, `vlan-id`, `gateway`, `gateway6`, `cidr`, `cidr6`, `mtu`.
Le GET renvoie aussi `address`/`netmask` (forme canonique : `cidr`), `bridge_stp`,
`bridge_fd`, `bond_miimon` (écrits par défaut par PVE), `options` (lignes inconnues de
l'API, jamais écrites), `active`, `exists`, `families`, `method`, `priority` : non répliqués.
- le PUT d'une interface recalcule `method` et `families` à partir des paramètres reçus :
sans `address`/`cidr`, une interface statique devient `manual`. pve-confsync envoie donc
toute la configuration de l'interface à chaque mise à jour.
- `interfaces.d/*` n'est pas lu par l'API ([Interfaces.pm](https://github.com/proxmox/pve-common/blob/master/src/PVE/Network/Interfaces.pm)) : répliqué en fichier par SSH.
Scripts SSH (première ligne `# confsync:<étape>`) : lecture (`stat`, `base64 -w0`),
préparation (contrôle SHA-256, `cp -p` vers la sauvegarde, `restore.sh`, écriture par
`mktemp` + `mv`), restauration, `systemd-run --on-active=120s` / `systemctl stop <unité>.timer`,
rechargements. Le script de préparation est testé tel quel avec le shell local
(`internal/nodesync/script_test.go`) ; **le reste est à valider sur les clusters de labo.**
## Droits
Accès en `root@pam` (API + SSH, même mot de passe), sans TFA. La référence (primaire)
n'est jamais écrite. Le certificat TLS et la clé d'hôte SSH sont épinglés au premier test.
docs/proposition-replication-noeuds.md
+117
View File
@@ -0,0 +1,117 @@
# Proposition : réplication de fichiers système nœud à nœud (SSH)
Statut : validée par Cédric le 2026-10-23 (liste v1, API dès que possible, identifiants
SSH hérités du cluster avec surcharge par nœud), puis implémentée. Écart avec la
proposition : `/etc/network/interfaces` passe par l'API PVE (`/nodes/{node}/network`) et
non par fichier, et le DNS est ajouté par l'API (`/nodes/{node}/dns`). Le fonctionnement
réel est décrit dans le README (« Réplication de nœuds »).
Deuxième mode de réplication, séparé de la réplication cluster à cluster (API) :
un **nœud maître** et un ou plusieurs **nœuds cibles** (même cluster ou non),
fichiers lus et écrits en SSH root. Même moteur : lecture, normalisation, diff,
plan, apply, vérification.
Constat dans le code actuel : l'accès SSH est défini **par cluster, sur un seul
hôte** (`SSHHost`, `internal/config/config.go:33`). Il faut le passer **par
nœud** : nœuds découverts par l'API (`GET /cluster/status`, champ `ip`), identifiants
root hérités du cluster et surchargeables par nœud, clé d'hôte épinglée par nœud
(même TOFU que maintenant).
## 1. Fichiers candidats
Légende : 🟢 v1 recommandée, 🟡 v1 possible avec garde-fous, 🔴 exclu.
| Fichier | Intérêt | Risques | Propre au nœud (mapper / exclure) | Rechargement |
|---|---|---|---|---|
| 🟢 `/etc/network/interfaces` | bonds, VLAN, bridges, MTU identiques partout | **perte d'accès au nœud** ; noms de NIC différents d'un nœud à l'autre | `address`, `gateway`, `hwaddress`, IP de management, éventuellement noms de NIC (`nic1` maître ↔ `enp65s0f0` cible) | `ifreload -a` |
| 🟢 `/etc/network/interfaces.d/*` | idem | **`interfaces.d/sdn` est généré par le SDN PVE** | exclure `sdn` d'office | `ifreload -a` |
| 🟢 `/etc/multipath.conf` | `defaults`, `blacklist`, `devices` identiques sur tout le cluster | erreur de syntaxe = chemins perdus | rien en général | `multipathd reconfigure` |
| 🟢 `/etc/multipath/wwids` | mêmes LUN vues par tous les nœuds | ajouter un WWID local (disque système) par erreur | mode additif seulement, jamais de suppression | `multipathd reconfigure` |
| 🟡 `/etc/multipath/bindings` | alias `mpathX` identiques sur tous les nœuds (important si LVM partagé référence les noms) | renommer un alias en service | additif seulement, conflit d'alias = bloquant | `multipathd reconfigure` |
| 🟢 `/etc/chrony/chrony.conf` (+ `sources.d`) | mêmes sources NTP | faible | rien | `systemctl restart chrony` |
| 🟡 `/etc/systemd/timesyncd.conf` | si chrony absent | faible | rien | `systemctl restart systemd-timesyncd` |
| 🟢 `/etc/sysctl.d/*.conf` | tuning réseau / mémoire homogène | réglage inadapté au matériel | rien ; ne jamais toucher les fichiers des paquets (`/usr/lib/sysctl.d`) | `sysctl --system` |
| 🟡 `/etc/modprobe.d/*.conf` | options ZFS (`zfs_arc_max`), blacklist de modules | effet au reboot seulement ; ARC dépend de la RAM | `zfs_arc_max` à surcharger par nœud | `update-initramfs -u`, **redémarrage requis** (signalé, jamais fait) |
| 🟡 `/etc/apt/sources.list.d/*.sources` | mêmes dépôts (enterprise / no-subscription / ceph) | dépôt enterprise sans abonnement sur la cible | rien | aucun (pas d'`apt update` automatique) |
| 🟡 `/etc/lvm/lvm.conf` | `global_filter` pour iSCSI / multipath | **filtre faux = volumes invisibles** | rien en général | aucun ; vérification `pvs` avant / après |
| 🟡 `/etc/iscsi/iscsid.conf` | timeouts, `node.startup` | **contient les secrets CHAP** | secrets chiffrés au repos, jamais affichés en clair dans le diff | `systemctl restart iscsid` (signalé) |
| 🟡 `/etc/vzdump.conf` | défauts de sauvegarde par nœud | faible | `tmpdir` éventuellement | aucun |
| 🟡 `/etc/postfix/main.cf` | relais SMTP des notifications | faible | `myhostname` | `postfix reload` |
| 🔵 `/etc/resolv.conf` | DNS identiques | écrasé par DHCP / resolvconf | rien | aucun. **Existe aussi en API** (`/nodes/{node}/dns`), à préférer : à vérifier dans l'API viewer |
| 🔵 `/etc/hosts` | résolution des nœuds entre eux | ligne du nœud lui-même ; corosync dépend du nom | ligne `127.0.1.1` et la ligne du nœud cible | aucun. **Existe aussi en API** (`/nodes/{node}/hosts`), à vérifier |
| 🔴 `/etc/pve/*` (dont `corosync.conf`) | déjà partagé par pmxcfs dans un cluster | casser le cluster | | |
| 🔴 `/etc/corosync/*`, `/etc/hostname`, `/etc/machine-id`, clés SSH d'hôte, `/etc/iscsi/initiatorname.iscsi` | | identité unique du nœud | | |
| 🔴 `/etc/frr/*` | | géré par le SDN EVPN (hors périmètre) | | |
| 🔴 `/etc/kernel/cmdline`, `/etc/default/grub` | | démarrage du nœud | | v2 éventuellement |
🔵 = je propose de passer par l'API PVE plutôt que par le fichier (déjà le modèle du reste de l'outil).
**Ma recommandation v1 :** `interfaces` (+ `interfaces.d` hors `sdn`), `multipath.conf`, `wwids`, `chrony.conf`, `sysctl.d`. Le reste en v1.1 une fois le mécanisme éprouvé.
## 2. Modèle
**Profil de réplication** : un nœud maître, des nœuds cibles, une liste de fichiers, et par fichier des règles.
**Règles par fichier**, deux niveaux :
- **Adaptateur structuré** pour `interfaces` (stanzas `iface`) et `multipath.conf` (sections) : politique par champ comme pour l'API (sync / ignorer / surcharger par cible). Exemple : `iface vmbr0 / address` = surcharger, valeur par cible.
- **Mode texte** pour les autres : diff ligne à ligne, avec des **lignes protégées** (regex) conservées telles quelles sur la cible, et des **variables** (`{{hostname}}`, `{{ip}}`, valeurs saisies par cible).
- **Mapping de noms de NIC** par cible (`eno1` → `enp1s0f0`), appliqué avant le diff.
**Déroulé :**
1. **Lecture** : `cat` + `sha256sum` + propriétaire et droits, sur le maître et chaque cible.
2. **Diff** : diff unifié par fichier et par cible, après application des règles. Statut : à jour / écarts / bloquant (ex. NIC du maître absente sur la cible, alias multipath en conflit).
3. **Dry-run** (par défaut) : rien n'est écrit, on affiche le fichier final exact et la commande de rechargement.
4. **Apply**, après **confirmation explicite** (comme le SDN), **un nœud à la fois**, arrêt au premier échec :
- le hash de la cible doit être celui lu au plan (sinon : modifié entre-temps, on refuse) ;
- **sauvegarde** sur le nœud (`/var/lib/pve-confsync/backup/<chemin>.<horodatage>`) et en base (SQLite, historique) ;
- **validation** avant écriture quand l'outil existe : `multipath -t` sur un fichier temporaire, contrôle de syntaxe ifupdown2 (`ifup --no-act`, option exacte à vérifier) ;
- écriture **atomique** (fichier temporaire + `mv`), propriétaire et droits conservés ;
- **rechargement** (colonne de droite du tableau), seulement si coché ;
- **vérification** : relecture du hash, et pour le réseau, reconnexion SSH.
5. **Filet de sécurité réseau** : avant `ifreload -a`, on programme sur le nœud une restauration automatique (`systemd-run --on-active=120s` qui remet la sauvegarde et relance `ifreload -a`). Si l'outil se reconnecte en SSH dans le délai, il annule la restauration. Sinon le nœud revient seul à l'ancienne config.
6. **Suppression** : jamais par défaut. Un fichier absent du maître n'est pas supprimé sur la cible sauf option explicite (cohérent avec le mode additif du reste).
**Journal et audit** : chaque exécution (profil, nœud, fichier, hash avant / après, sauvegarde, sortie du rechargement) dans le journal existant, avec un type « Nœuds ».
**Droits** : opérateur = dry-run, admin = apply (le SSH root est le droit le plus fort de l'outil).
## 3. Maquette de l'interface
Un sélecteur en tête sépare les deux modes. Chaque mode garde les onglets Écarts / Profils (ou Clusters) / Journal.
```
┌──────────────────────────────────────────────────────────────────────┐
│ pve-confsync [ Clusters (API) | Nœuds (fichiers) ] 👤 ⏻ │
├──────────────────────────────────────────────────────────────────────┤
│ [Écarts] [Profils] [Journal] Profil: [Réseau stockage ▾] 2 écarts │
│ [Dry-run global] [Appliquer]│
├────────────────────────────┬─────────────────────────────────────────┤
│ MAÎTRE pve1 (lab-a) │ pve2 · /etc/network/interfaces ÉCARTS │
│ │ │
│ CIBLES │ auto bond0 │
│ ┌────────────────────────┐ │ iface bond0 inet manual │
│ │ pve2 (lab-a) ÉCARTS │ │ - bond-mode active-backup │
│ │ interfaces ✎ 3 │ │ + bond-mode 802.3ad │
│ │ multipath.conf ✓ │ │ + bond-xmit-hash-policy layer3+4 │
│ │ chrony.conf ✓ │ │ iface vmbr0 inet static │
│ └────────────────────────┘ │ address 10.0.0.12/24 🔒 surchargé │
│ ┌────────────────────────┐ │ bridge-ports bond0 │
│ │ pve3 (lab-a) À JOUR │ │ │
│ └────────────────────────┘ │ Après écriture : ifreload -a ☑ │
│ ┌────────────────────────┐ │ Restauration auto si perte SSH : 120 s │
│ │ pve7 (lab-b) BLOQUANT │ │ │
│ │ interfaces NIC eno2 │ │ [Dry-run] [Appliquer sur pve2] │
│ │ absente, à mapper │ │ │
│ └────────────────────────┘ │ │
└────────────────────────────┴─────────────────────────────────────────┘
```
Onglet **Profils** : maître, cibles (nœuds découverts par cluster, cases à cocher), fichiers (cases parmi la liste du §1), règles par fichier (champs à surcharger, lignes protégées, mapping de NIC par cible).
Fenêtre de **confirmation** d'apply : liste des nœuds dans l'ordre, fichiers, commandes de rechargement, avertissements (« redémarrage requis », « accès réseau »), saisie du nom du nœud pour confirmer quand `interfaces` est concerné.
## Questions ouvertes
1. Liste v1 : celle recommandée ci-dessus, ou plus large ?
2. `resolv.conf` et `hosts` par l'API plutôt que par fichier : d'accord ?
3. Identifiants SSH : hérités du cluster avec surcharge par nœud, d'accord ?
examples/confsync.example.yaml
-63
View File
@@ -1,63 +0,0 @@
# pve-confsync - exemple de configuration (phase 0, CLI)
# Les secrets des tokens ne sont JAMAIS écrits ici : variable d'environnement
# (token_secret_env) ou fichier, par exemple un secret docker (token_secret_file).
reference:
name: ref
url: https://pve-ref.lab:8006
token_id: confsync@pve!ro # token en lecture seule (Sys.Audit, SDN.Audit)
token_secret_file: /run/secrets/pve_ref # secret docker (voir docker-compose.yml)
fingerprint: "AA:BB:CC:...:FF" # SHA-256 du certificat (pinning, certificat auto-signé)
clients:
- name: site-a
url: https://pve-a.lab:8006
token_id: confsync@pve!rw
token_secret_file: /run/secrets/pve_site_a
ca_file: /config/ca-site-a.pem # ou fingerprint, ou insecure: true (labo uniquement)
# Valeurs propres au cluster, prioritaires sur la référence :
# type d'objet -> id -> champ -> valeur
mapping:
sdn-zone:
vlanz:
bridge: vmbr1 # obligatoire pour créer une zone VLAN
nodes: pve-a1,pve-a2
vxz:
peers: 192.168.10.1,192.168.10.2
datacenter-options:
datacenter:
next-id: lower=2000,upper=2999
# Politiques propres à ce client (prioritaires sur les politiques globales)
policies:
sdn-zone:
mtu: ignore
- name: site-b
url: https://pve-b.lab:8006
token_id: confsync@pve!rw
token_secret_env: PVE_SITE_B_SECRET
insecure: true
mapping:
sdn-zone:
vlanz:
bridge: vmbr0
# Types synchronisés (défaut : tous)
kinds: [datacenter-options, sdn-zone, sdn-vnet, sdn-subnet]
# Sélection des objets par motif (glob) sur l'id
objects:
sdn-zone:
exclude: ["lab*"]
# Politique par champ : sync | ignore | override (valeur fournie par le mapping du client)
# Défauts : options datacenter = tag-style, registered-tags, user-tag-access, next-id en sync,
# le reste ignoré ; zones SDN = nodes, bridge, peers, fabric, dns, reversedns, dnszone en
# override, le reste en sync ; vnets et subnets = sync.
policies:
datacenter-options:
keyboard: sync
language: sync
# Suppression chez les clients des objets absents de la référence (défaut : non)
prune: false
go.mod
+2 -2
View File
@@ -3,7 +3,7 @@ module pve-confsync
go 1.24.0
require (
gopkg.in/yaml.v3 v3.0.1
golang.org/x/crypto v0.43.0
modernc.org/sqlite v1.40.0
)
@@ -14,7 +14,7 @@ require (
github.com/ncruces/go-strftime v0.1.9 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b // indirect
golang.org/x/sys v0.36.0 // indirect
golang.org/x/sys v0.37.0 // indirect
modernc.org/libc v1.66.10 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
go.sum
+6 -6
View File
@@ -10,6 +10,8 @@ github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdh
github.com/ncruces/go-strftime v0.1.9/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
golang.org/x/crypto v0.43.0 h1:dduJYIi3A3KOfdGOHX8AVZ/jGiyPa3IbBozJ5kNuE04=
golang.org/x/crypto v0.43.0/go.mod h1:BFbav4mRNlXJL4wNeejLpWxB7wMbc79PdRGhWKncxR0=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b h1:M2rDM6z3Fhozi9O7NWsxAkg/yqS/lQJ6PmkyIV3YP+o=
golang.org/x/exp v0.0.0-20250620022241-b7579e27df2b/go.mod h1:3//PLf8L/X+8b4vuAfHzxeRUl04Adcb341+IGKfnqS8=
golang.org/x/mod v0.27.0 h1:kb+q2PyFnEADO2IEF935ehFUXlWiNjJWtRNgBLSfbxQ=
@@ -17,14 +19,12 @@ golang.org/x/mod v0.27.0/go.mod h1:rWI627Fq0DEoudcK+MBkNkCe0EetEaDSwJJkCcjpazc=
golang.org/x/sync v0.16.0 h1:ycBJEhp9p4vXvUZNszeOq0kGTPghopOL8q0fq3vstxw=
golang.org/x/sync v0.16.0/go.mod h1:1dzgHSNfp02xaA81J2MS99Qcpr2w7fw1gpm99rleRqA=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/sys v0.37.0 h1:fdNQudmxPjkdUTPnLn5mdQv7Zwvbvpaxqs831goi9kQ=
golang.org/x/sys v0.37.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/term v0.36.0 h1:zMPR+aF8gfksFprF/Nc/rd1wRS1EI6nDBGyWAvDzx2Q=
golang.org/x/term v0.36.0/go.mod h1:Qu394IJq6V6dCBRgwqshf3mPF85AqzYEzofzRdZkWss=
golang.org/x/tools v0.36.0 h1:kWS0uv/zsvHEle1LbV5LE8QujrxB3wfQyxHfhOk0Qkg=
golang.org/x/tools v0.36.0/go.mod h1:WBDiHKJK8YgLHlcQPYQzNCkUxUypCaa5ZegCVutKm+s=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
modernc.org/cc/v4 v4.26.5 h1:xM3bX7Mve6G8K8b+T11ReenJOT+BmVqQj0FY5T4+5Y4=
modernc.org/cc/v4 v4.26.5/go.mod h1:uVtb5OGqUKpoLWhqwNQo/8LwvoiEBLvZXIQ/SmO6mL0=
modernc.org/ccgo/v4 v4.28.1 h1:wPKYn5EC/mYTqBO373jKjvX2n+3+aK7+sICCv4Fjy1A=
internal/adapter/access.go
+345
View File
@@ -0,0 +1,345 @@
package adapter
import (
"context"
"errors"
"fmt"
"net/url"
"sort"
"strings"
"pve-confsync/internal/model"
"pve-confsync/internal/pve"
)
// Access control objects of user.cfg, through /access (pve-access-control,
// src/PVE/API2/Group.pm, Role.pm, User.pm). Only the accounts of the pve realm
// (Proxmox VE authentication server) are replicated: pam accounts are Linux
// accounts of each node, the other realms (LDAP, AD, OpenID) are synchronized
// from their directory.
// ---- groups: /access/groups ----
type group struct{}
func (group) Kind() model.Kind { return model.KindGroup }
func (group) SDN() bool { return false }
func (group) Singleton() bool { return false }
func (group) List(ctx context.Context, c *pve.Client, _ ReadMode, _ []model.Object) ([]model.Object, []string, error) {
var raw []map[string]any
if err := c.Get(ctx, "access/groups", nil, &raw); err != nil {
return nil, nil, err
}
var objs []model.Object
for _, r := range raw {
// members come from the "groups" field of the accounts
f := dropEmpty(normalizeAPI(r, "groupid", "users"))
objs = append(objs, model.Object{Kind: model.KindGroup, ID: pve.Str(r["groupid"]), Fields: f})
}
return objs, nil, nil
}
func (group) DefaultPolicy(string) Policy { return Sync }
func (group) Canonicalize(map[string]any) {}
func (group) Forbidden(f string) bool { return f == "users" }
func (group) Immutable(string) bool { return false }
func (group) Required(model.Object) []string { return nil }
func (group) Create(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
f := form(o.Fields, nil, "")
f.Set("groupid", o.ID)
return c.Post(ctx, "access/groups", f, nil)
}
func (group) Update(ctx context.Context, c *pve.Client, t model.Object, set map[string]any, unset []string, _ string) error {
return c.Put(ctx, "access/groups/"+url.PathEscape(t.ID), form(blank(set, unset), nil, ""), nil)
}
func (group) Delete(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
return c.Delete(ctx, "access/groups/"+url.PathEscape(o.ID), nil, nil)
}
// ---- roles: /access/roles (custom roles only) ----
type role struct{}
func (role) Kind() model.Kind { return model.KindRole }
func (role) SDN() bool { return false }
func (role) Singleton() bool { return false }
func (role) List(ctx context.Context, c *pve.Client, _ ReadMode, _ []model.Object) ([]model.Object, []string, error) {
var raw []map[string]any
if err := c.Get(ctx, "access/roles", nil, &raw); err != nil {
return nil, nil, err
}
var objs []model.Object
for _, r := range raw {
if pve.Str(r["special"]) == "1" { // built-in roles (Administrator, PVEAuditor…)
continue
}
f := normalizeAPI(r, "roleid", "special")
role{}.Canonicalize(f)
objs = append(objs, model.Object{Kind: model.KindRole, ID: pve.Str(r["roleid"]), Fields: f})
}
return objs, nil, nil
}
func (role) DefaultPolicy(string) Policy { return Sync }
func (role) Canonicalize(f map[string]any) { sortList(f, "privs") }
func (role) Forbidden(string) bool { return false }
func (role) Immutable(string) bool { return false }
func (role) Required(model.Object) []string { return nil }
func (role) Create(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
f := form(o.Fields, nil, "")
f.Set("roleid", o.ID)
return c.Post(ctx, "access/roles", f, nil)
}
// Update replaces the privileges (no "append").
func (role) Update(ctx context.Context, c *pve.Client, t model.Object, set map[string]any, unset []string, _ string) error {
return c.Put(ctx, "access/roles/"+url.PathEscape(t.ID), form(blank(set, unset), nil, ""), nil)
}
func (role) Delete(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
return c.Delete(ctx, "access/roles/"+url.PathEscape(o.ID), nil, nil)
}
// ---- accounts of the pve realm: /access/users ----
// PVERealm is the realm whose accounts are replicated (type "pve").
const PVERealm = "pve"
type user struct{}
func (user) Kind() model.Kind { return model.KindUser }
func (user) SDN() bool { return false }
func (user) Singleton() bool { return false }
func (user) List(ctx context.Context, c *pve.Client, _ ReadMode, _ []model.Object) ([]model.Object, []string, error) {
var raw []map[string]any
if err := c.Get(ctx, "access/users", url.Values{"full": {"1"}}, &raw); err != nil {
return nil, nil, err
}
var objs []model.Object
for _, r := range raw {
id := pve.Str(r["userid"])
if !strings.HasSuffix(id, "@"+PVERealm) {
continue
}
// tokens hold no secret here, but are specific to each cluster (their secret is)
f := dropEmpty(normalizeAPI(r, "userid", "tokens", "realm-type", "totp-locked", "tfa-locked-until"))
user{}.Canonicalize(f)
objs = append(objs, model.Object{Kind: model.KindUser, ID: id, Fields: f})
}
return objs, nil, nil
}
func (user) DefaultPolicy(string) Policy { return Sync }
// Canonicalize: "groups" is a comma separated list; expire 0 means no expiration (absent).
func (user) Canonicalize(f map[string]any) {
if l, ok := f["groups"].([]string); ok {
f["groups"] = strings.Join(l, ",")
}
sortList(f, "groups")
if f["expire"] == "0" {
delete(f, "expire")
}
}
// Forbidden: the password is never set through the API (only its hash is copied, see password).
func (user) Forbidden(f string) bool { return f == "password" || f == "tokens" }
func (user) Immutable(string) bool { return false }
func (user) Required(model.Object) []string { return nil }
func (user) Parents(o model.Object) []string {
g, _ := o.Fields["groups"].(string)
var out []string
for _, x := range strings.Split(g, ",") {
if x = strings.TrimSpace(x); x != "" {
out = append(out, string(model.KindGroup)+"/"+x)
}
}
return out
}
func (user) Create(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
f := form(o.Fields, nil, "")
f.Set("userid", o.ID)
return c.Post(ctx, "access/users", f, nil)
}
// Update: PUT /access/users/{userid} has no "delete" parameter; a removed field
// is written empty (expire 0, groups "" removes every membership).
func (user) Update(ctx context.Context, c *pve.Client, t model.Object, set map[string]any, unset []string, _ string) error {
f := blank(set, unset)
if _, ok := f["expire"]; ok && f["expire"] == "" {
f["expire"] = "0"
}
if _, ok := f["enable"]; ok && f["enable"] == "" {
f["enable"] = "1"
}
return c.Put(ctx, "access/users/"+url.PathEscape(t.ID), form(f, nil, ""), nil)
}
func (user) Delete(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
if o.ID == "root@pam" { // never, whatever the selection
return errors.New("root@pam cannot be deleted")
}
return c.Delete(ctx, "access/users/"+url.PathEscape(o.ID), nil, nil)
}
// ---- password hashes: /etc/pve/priv/shadow.cfg over SSH ----
// The API only takes clear passwords (PUT /access/password); the hashes are
// copied as they are, so the accounts keep their password on every cluster.
// Format (src/PVE/Auth/PVE.pm): one "user:hash:" line per account, the user
// without its realm. Writes go through the pmxcfs lock of the file, like PVE
// (PVE::Auth::PVE::lock_shadow_config, cfs_read_file / cfs_write_file).
type password struct{}
const shadowRead = "cat /etc/pve/priv/shadow.cfg 2>/dev/null || true"
// shadowWrite reads "set\tuser\thash" / "del\tuser" lines on stdin: the hashes
// never appear on a command line.
const shadowWrite = `perl -e '
use strict; use warnings;
use PVE::Cluster;
use PVE::Auth::PVE;
my @ops;
while (my $l = <STDIN>) { chomp $l; push @ops, [split(/\t/, $l, 3)]; }
PVE::Auth::PVE::lock_shadow_config(sub {
my $cfg = PVE::Cluster::cfs_read_file("priv/shadow.cfg");
for my $o (@ops) {
my ($op, $u, $h) = @$o;
die "invalid user\n" if !defined($u) || $u !~ m/^[^\s:]+$/;
if ($op eq "set") {
die "invalid hash\n" if !defined($h) || $h !~ m/^[^\s:]+$/;
$cfg->{users}->{$u}->{shadow} = $h;
} elsif ($op eq "del") {
delete $cfg->{users}->{$u};
} else {
die "invalid operation\n";
}
}
PVE::Cluster::cfs_write_file("priv/shadow.cfg", $cfg);
}, "confsync");
print "ok\n";
' # confsync-shadow-write`
func (password) Kind() model.Kind { return model.KindPassword }
func (password) SDN() bool { return false }
func (password) Singleton() bool { return false }
// List reads the hashes of the known accounts of the pve realm (the accounts
// are read before, see KindDeps): orphan lines are left aside.
func (password) List(ctx context.Context, c *pve.Client, _ ReadMode, known []model.Object) ([]model.Object, []string, error) {
sh, err := c.Shell()
if err != nil {
return nil, nil, err
}
out, err := sh.Run(ctx, shadowRead, nil)
if err != nil {
return nil, nil, err
}
users := ids(known, model.KindUser)
var objs []model.Object
for _, line := range strings.Split(string(out), "\n") {
line = strings.TrimSpace(line)
p := strings.Split(line, ":")
if len(p) != 3 || p[0] == "" || p[1] == "" {
continue
}
id := p[0] + "@" + PVERealm
if !users[id] {
continue
}
objs = append(objs, model.Object{Kind: model.KindPassword, ID: id, Fields: map[string]any{"hash": p[1]}})
}
sort.Slice(objs, func(i, j int) bool { return objs[i].ID < objs[j].ID })
return objs, nil, nil
}
func (password) DefaultPolicy(string) Policy { return Sync }
func (password) Canonicalize(map[string]any) {}
func (password) Forbidden(f string) bool { return f != "hash" }
func (password) Immutable(string) bool { return false }
func (password) Required(model.Object) []string { return []string{"hash"} }
func (password) Sensitive(f string) bool { return f == "hash" }
func (password) Parents(o model.Object) []string {
return []string{string(model.KindUser) + "/" + o.ID}
}
func shadowUser(id string) (string, error) {
u, ok := strings.CutSuffix(id, "@"+PVERealm)
if !ok || u == "" {
return "", fmt.Errorf("%s: not an account of the %s realm", id, PVERealm)
}
return u, nil
}
func (password) write(ctx context.Context, c *pve.Client, line string) error {
sh, err := c.Shell()
if err != nil {
return err
}
out, err := sh.Run(ctx, shadowWrite, []byte(line+"\n"))
if err != nil {
return err
}
if strings.TrimSpace(string(out)) != "ok" {
return errors.New("shadow.cfg: unexpected answer")
}
return nil
}
func (p password) Create(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
return p.Update(ctx, c, o, o.Fields, nil, "")
}
func (p password) Update(ctx context.Context, c *pve.Client, t model.Object, set map[string]any, _ []string, _ string) error {
u, err := shadowUser(t.ID)
if err != nil {
return err
}
h, _ := set["hash"].(string)
if h == "" {
return errors.New("empty password hash")
}
return p.write(ctx, c, "set\t"+u+"\t"+h)
}
func (p password) Delete(ctx context.Context, c *pve.Client, o model.Object, _ string) error {
u, err := shadowUser(o.ID)
if err != nil {
return err
}
return p.write(ctx, c, "del\t"+u)
}
// ---- helpers ----
// dropEmpty removes empty values: PVE returns some optional fields empty.
func dropEmpty(f map[string]any) map[string]any {
for k, v := range f {
if v == "" {
delete(f, k)
}
}
return f
}
// blank merges the fields to remove as empty values (APIs without "delete").
func blank(set map[string]any, unset []string) map[string]any {
out := make(map[string]any, len(set)+len(unset))
for k, v := range set {
out[k] = v
}
for _, k := range unset {
out[k] = ""
}
return out
}
internal/adapter/adapter.go
+85 -1
View File
@@ -5,6 +5,8 @@ package adapter
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"net/url"
"sort"
@@ -60,7 +62,89 @@ type Adapter interface {
// All returns the adapters in dependency order.
func All() []Adapter {
return []Adapter{datacenter{}, zone{}, vnet{}, subnet{}}
return []Adapter{datacenter{}, group{}, role{}, user{}, password{}, zone{}, vnet{}, subnet{}}
}
// KindDeps: replicating a kind needs these kinds too (a password needs its
// account on the target, an account its groups).
var KindDeps = map[model.Kind][]model.Kind{
model.KindPassword: {model.KindUser},
model.KindUser: {model.KindGroup},
}
// KindBundles are kinds chosen as one item: the accounts of the pve realm go
// with their passwords (the API holds the accounts, shadow.cfg the passwords).
var KindBundles = [][]model.Kind{{model.KindUser, model.KindPassword}}
// Bundle adds to a list of kinds the other kinds of their bundles, in the
// order of model.Kinds.
func Bundle(kinds []model.Kind) []model.Kind {
in := map[model.Kind]bool{}
for _, k := range kinds {
in[k] = true
}
for _, b := range KindBundles {
for _, k := range b {
if in[k] {
for _, x := range b {
in[x] = true
}
break
}
}
}
out := []model.Kind{}
for _, k := range model.Kinds {
if in[k] {
out = append(out, k)
}
}
return out
}
// Sensitive is implemented by adapters whose fields must never be shown nor
// journaled (password hashes): they are replaced by Redact.
type Sensitive interface {
Sensitive(field string) bool
}
// Parented is implemented by adapters whose objects need other objects on the
// target ("kind/id" keys): an account needs its groups, a password its account.
type Parented interface {
Parents(o model.Object) []string
}
// IsSensitive tells whether a field of a kind is secret.
func IsSensitive(k model.Kind, field string) bool {
a, err := ByKind(k)
if err != nil {
return false
}
s, ok := a.(Sensitive)
return ok && s.Sensitive(field)
}
// Redact replaces a secret value by a short digest: two different hashes stay
// distinguishable in a diff, the value itself is never shown.
func Redact(v any) any {
s, ok := v.(string)
if !ok || s == "" {
return v
}
sum := sha256.Sum256([]byte(s))
return "<hidden " + hex.EncodeToString(sum[:3]) + ">"
}
// Parents returns the keys of the objects o needs on the target.
func Parents(o model.Object) []string {
a, err := ByKind(o.Kind)
if err != nil {
return nil
}
if p, ok := a.(Parented); ok {
return p.Parents(o)
}
return nil
}
// ByKind returns the adapter of a kind.
internal/config/config.go
+71 -82
View File
@@ -1,81 +1,62 @@
// Package config loads the YAML configuration of the CLI (phase 0): clusters,
// field policies, per client mappings and object selection.
// Token secrets are never written in this file: they are read from an
// environment variable or a file (docker secret), see README.
// Package config is the synchronization configuration given to the engine:
// the primary cluster, the targets with their mappings and policies, the
// replicated kinds and the object selection. It is built from the database of
// the web UI (internal/store); credentials are decrypted only in memory.
package config
import (
"errors"
"fmt"
"os"
"net"
"net/url"
"path"
"strings"
"time"
"gopkg.in/yaml.v3"
"pve-confsync/internal/adapter"
"pve-confsync/internal/model"
"pve-confsync/internal/pve"
"pve-confsync/internal/sshx"
)
// Cluster is the connection to one PVE cluster.
// Cluster is the connection to one PVE cluster: the API with the root@pam
// credentials and SSH as root on one node for what the API does not cover.
type Cluster struct {
Name string `yaml:"name"`
URL string `yaml:"url"`
TokenID string `yaml:"token_id"`
TokenSecretEnv string `yaml:"token_secret_env,omitempty"`
TokenSecretFile string `yaml:"token_secret_file,omitempty"`
CAFile string `yaml:"ca_file,omitempty"`
Fingerprint string `yaml:"fingerprint,omitempty"`
Insecure bool `yaml:"insecure,omitempty"`
Timeout string `yaml:"timeout,omitempty"`
// Set by the web UI from its database, never read from nor written to YAML.
Secret string `yaml:"-"` // token secret, decrypted
CAPEM string `yaml:"-"` // CA bundle, PEM
Name string
URL string
User string // API user, root@pam
Password string // decrypted, never logged
Fingerprint string // pinned TLS certificate (SHA-256)
CAPEM string // CA bundle, PEM
Insecure bool
Timeout string
SSHHost string // empty: the host of URL
SSHPort int
SSHHostKey string // pinned host key (authorized_keys format)
}
// Client is a target cluster with its mapping (kind -> id -> field -> value)
// and its own policy overrides.
type Client struct {
Cluster `yaml:",inline"`
Mapping map[model.Kind]map[string]map[string]any `yaml:"mapping,omitempty"`
Policies map[model.Kind]map[string]adapter.Policy `yaml:"policies,omitempty"`
Cluster
Mapping map[model.Kind]map[string]map[string]any
Policies map[model.Kind]map[string]adapter.Policy
}
// Selection filters the objects of a kind with glob patterns on their id.
type Selection struct {
Include []string `yaml:"include,omitempty"`
Exclude []string `yaml:"exclude,omitempty"`
Include []string `json:"include,omitempty"`
Exclude []string `json:"exclude,omitempty"`
}
// Config is the whole file.
// Config is the whole synchronization: Reference is the primary cluster.
type Config struct {
Reference Cluster `yaml:"reference"`
Clients []Client `yaml:"clients"`
Kinds []model.Kind `yaml:"kinds,omitempty"`
Objects map[model.Kind]Selection `yaml:"objects,omitempty"`
Policies map[model.Kind]map[string]adapter.Policy `yaml:"policies,omitempty"`
Prune bool `yaml:"prune,omitempty"`
}
// Load reads and validates a configuration file.
func Load(file string) (*Config, error) {
raw, err := os.ReadFile(file)
if err != nil {
return nil, err
}
var c Config
dec := yaml.NewDecoder(strings.NewReader(string(raw)))
dec.KnownFields(true) // typos must fail, not be silently ignored
if err := dec.Decode(&c); err != nil {
return nil, fmt.Errorf("%s: %w", file, err)
}
if len(c.Kinds) == 0 {
c.Kinds = append(c.Kinds, model.Kinds...)
}
return &c, c.validate()
Reference Cluster
Clients []Client
Kinds []model.Kind
Objects map[model.Kind]Selection
Policies map[model.Kind]map[string]adapter.Policy
Prune bool
}
// Validate checks a configuration built elsewhere (web UI database, YAML import).
@@ -93,14 +74,16 @@ func (c *Config) validate() error {
errs = append(errs, fmt.Errorf("cluster %s: duplicate name", cl.Name))
}
seen[cl.Name] = true
if cl.Secret == "" && cl.TokenSecretEnv == "" && cl.TokenSecretFile == "" {
errs = append(errs, fmt.Errorf("cluster %s: token_secret_env or token_secret_file is required", cl.Name))
}
}
for _, k := range c.Kinds {
if _, err := adapter.ByKind(k); err != nil {
errs = append(errs, err)
}
for _, d := range adapter.KindDeps[k] {
if !c.HasKind(d) {
errs = append(errs, fmt.Errorf("%s needs %s", k, d))
}
}
}
check := func(where string, p map[model.Kind]map[string]adapter.Policy) {
for k, fields := range p {
@@ -151,45 +134,39 @@ func (c *Config) Client(name string) (*Client, bool) {
return nil, false
}
// PVE builds the API client of a cluster, resolving its token secret.
// PVE builds the API client of a cluster, with its SSH access when the host
// key is recorded.
func (cl Cluster) PVE() (*pve.Client, error) {
secret, err := cl.secret()
if err != nil {
return nil, err
}
var timeout time.Duration
var err error
if cl.Timeout != "" {
if timeout, err = time.ParseDuration(cl.Timeout); err != nil {
return nil, fmt.Errorf("cluster %s: timeout: %w", cl.Name, err)
}
}
return pve.New(pve.Options{
Name: cl.Name, BaseURL: cl.URL, TokenID: cl.TokenID, TokenSecret: secret,
CAFile: cl.CAFile, CAPEM: []byte(cl.CAPEM), Fingerprint: cl.Fingerprint, Insecure: cl.Insecure, Timeout: timeout,
})
o := pve.Options{
Name: cl.Name, BaseURL: cl.URL, User: cl.User, Password: cl.Password,
CAPEM: []byte(cl.CAPEM), Fingerprint: cl.Fingerprint, Insecure: cl.Insecure, Timeout: timeout,
}
if cl.SSHHostKey != "" {
sh, err := sshx.New(cl.SSHOptions())
if err != nil {
return nil, fmt.Errorf("cluster %s: %w", cl.Name, err)
}
o.Shell = sh
}
return pve.New(o)
}
// ResolveSecret reads the token secret from the environment variable or file of
// the YAML configuration (used when a YAML file is imported in the web UI).
func (cl Cluster) ResolveSecret() (string, error) { return cl.secret() }
func (cl Cluster) secret() (string, error) {
if cl.Secret != "" {
return cl.Secret, nil
}
if cl.TokenSecretEnv != "" {
if v := os.Getenv(cl.TokenSecretEnv); v != "" {
return v, nil
}
if cl.TokenSecretFile == "" {
return "", fmt.Errorf("cluster %s: environment variable %s is empty", cl.Name, cl.TokenSecretEnv)
// SSHOptions is the SSH access of the cluster: root on SSHHost, or on the host of the API URL.
func (cl Cluster) SSHOptions() sshx.Options {
host := cl.SSHHost
if host == "" {
if u, err := url.Parse(cl.URL); err == nil {
host = u.Hostname()
}
}
b, err := os.ReadFile(cl.TokenSecretFile)
if err != nil {
return "", fmt.Errorf("cluster %s: token secret: %w", cl.Name, err)
}
return strings.TrimSpace(string(b)), nil
return sshx.Options{Host: host, Port: cl.SSHPort, User: "root", Password: cl.Password, HostKey: cl.SSHHostKey}
}
// Policy resolves the policy of a field for a client: client policy, then
@@ -239,3 +216,15 @@ func (c *Config) HasKind(k model.Kind) bool {
}
return false
}
// NormalizeURL trims the URL of a cluster and adds the PVE port (8006) when
// none is given: https://pve1 becomes https://pve1:8006.
func NormalizeURL(raw string) string {
s := strings.TrimRight(strings.TrimSpace(raw), "/")
u, err := url.Parse(s)
if err != nil || u.Host == "" || u.Port() != "" {
return s
}
u.Host = net.JoinHostPort(u.Hostname(), "8006")
return u.String()
}
internal/config/config_test.go
+34 -25
View File
@@ -1,44 +1,53 @@
package config
import (
"os"
"path/filepath"
"strings"
"testing"
"pve-confsync/internal/adapter"
"pve-confsync/internal/model"
)
func TestLoadExample(t *testing.T) {
c, err := Load(filepath.Join("..", "..", "examples", "confsync.example.yaml"))
if err != nil {
t.Fatal(err)
func TestValidate(t *testing.T) {
c := &Config{
Reference: Cluster{Name: "ref"},
Clients: []Client{{Cluster: Cluster{Name: "ref"}}},
Kinds: []model.Kind{model.KindPassword},
Policies: map[model.Kind]map[string]adapter.Policy{model.KindDatacenter: {"u2f": adapter.Sync, "x": "bogus"}},
}
a, _ := adapter.ByKind(model.KindZone)
cl, _ := c.Client("site-a")
if p := c.Policy(cl, a, "mtu"); p != adapter.Ignore {
t.Fatalf("site-a mtu policy = %s", p)
err := c.Validate()
if err == nil {
t.Fatal("expected errors")
}
if p := c.Policy(nil, a, "bridge"); p != adapter.Override {
t.Fatalf("bridge policy = %s", p)
for _, want := range []string{"duplicate name", "cannot be synchronized", "unknown policy", "pve-password needs pve-user"} {
if !strings.Contains(err.Error(), want) {
t.Errorf("missing %q in %v", want, err)
}
if c.Selected(model.KindZone, "lab1") || !c.Selected(model.KindZone, "vlanz") {
t.Fatal("selection")
}
}
func TestRejects(t *testing.T) {
for name, body := range map[string]string{
"clear secret": "reference: {name: r, url: https://x, token_id: a@pve!b, token_secret: s}\n",
"no secret": "reference: {name: r, url: https://x, token_id: a@pve!b}\n",
"forbidden": "reference: {name: r, url: https://x, token_id: a@pve!b, token_secret_env: X}\npolicies: {datacenter-options: {webauthn: sync}}\n",
"bad policy": "reference: {name: r, url: https://x, token_id: a@pve!b, token_secret_env: X}\npolicies: {sdn-zone: {mtu: copy}}\n",
"unknown kind": "reference: {name: r, url: https://x, token_id: a@pve!b, token_secret_env: X}\nkinds: [firewall]\n",
func TestSSHOptions(t *testing.T) {
cl := Cluster{URL: "https://pve1.lab:8006", Password: "p"}
if o := cl.SSHOptions(); o.Host != "pve1.lab" || o.User != "root" {
t.Fatalf("%+v", o)
}
cl.SSHHost = "10.0.0.1"
if o := cl.SSHOptions(); o.Host != "10.0.0.1" {
t.Fatalf("%+v", o)
}
}
func TestNormalizeURL(t *testing.T) {
for in, want := range map[string]string{
"https://pve1": "https://pve1:8006",
" https://pve1/ ": "https://pve1:8006",
"https://pve1:443": "https://pve1:443",
"https://10.0.0.1": "https://10.0.0.1:8006",
"https://[fd00::1]": "https://[fd00::1]:8006",
"https://pve1:8006/": "https://pve1:8006",
} {
f := filepath.Join(t.TempDir(), "c.yaml")
_ = os.WriteFile(f, []byte(body), 0o600)
if _, err := Load(f); err == nil {
t.Errorf("%s: expected an error", name)
if got := NormalizeURL(in); got != want {
t.Errorf("NormalizeURL(%q) = %q, want %q", in, got, want)
}
}
}
internal/demo/demo.go
+95 -12
View File
@@ -1,16 +1,19 @@
// Package demo starts fake PVE clusters (pvetest) and fills a database with
// them, to try the web UI without a Proxmox cluster: a reference, two clients
// with differences (missing objects, different fields, a missing mapping) and
// a PVE 8 cluster that is refused.
// Package demo starts fake PVE clusters (pvetest: API and SSH) and fills a
// database with them, to try the web UI without a Proxmox cluster: a primary,
// two targets with differences (missing objects, different fields, accounts
// and passwords, a missing mapping) and a PVE 8 cluster that is refused.
package demo
import (
"context"
"crypto/sha256"
"encoding/hex"
"strings"
"pve-confsync/internal/adapter"
"pve-confsync/internal/model"
"pve-confsync/internal/nodesync"
"pve-confsync/internal/pve"
"pve-confsync/internal/pve/pvetest"
"pve-confsync/internal/store"
)
@@ -50,6 +53,14 @@ func Start(ctx context.Context, st *store.Store) (*Clusters, error) {
"dhcp-range": []any{"start-address=10.1.0.100,end-address=10.1.0.200"}}
ref.Pending.Subnets["vlanz-10.2.0.0-24"] = map[string]any{"vnet": "v200", "gateway": "10.2.0.1"}
ref.Commit()
ref.Groups["admins"] = map[string]any{"comment": "Administrateurs"}
ref.Groups["ops"] = map[string]any{"comment": "Exploitation"}
ref.Roles["ConfSyncOps"] = "Sys.Audit,VM.Audit,VM.PowerMgmt"
ref.Users["alice@pve"] = map[string]any{"enable": 1, "firstname": "Alice", "email": "alice@example.org", "groups": "admins"}
ref.Users["bob@pve"] = map[string]any{"enable": 1, "firstname": "Bob", "email": "bob@example.org", "groups": "ops", "expire": 1893456000}
ref.Users["carol@pam"] = map[string]any{"enable": 1, "comment": "Linux account, not replicated"}
ref.Shadow["alice"] = "$5$Kd1Xa9$Q8yJdemoAliceHashOnlyForTheDemo0123456789AB"
ref.Shadow["bob"] = "$5$Pq7Zr2$W3vLdemoBobHashOnlyForTheDemo0123456789ABCDE"
// site-a: tags and an MTU behind, the VXLAN zone and the dev network missing
a := pvetest.New("9.0.10")
@@ -65,6 +76,9 @@ func Start(ctx context.Context, st *store.Store) (*Clusters, error) {
a.Pending.VNets["ceph"] = map[string]any{"zone": "stor", "tag": 50, "alias": "ceph public"}
a.Pending.Subnets["vlanz-10.1.0.0-24"] = map[string]any{"vnet": "v100", "gateway": "10.1.0.254"}
a.Commit()
a.Groups["admins"] = map[string]any{"comment": "Admins"}
a.Users["alice@pve"] = map[string]any{"enable": 1, "firstname": "Alice", "email": "alice@old.example.org", "groups": "admins"}
a.Shadow["alice"] = "$5$OldOld$oldAliceHashOnSiteA0123456789ABCDEFGHIJKLMNOP"
// site-b: nearly in line, but no mapping for the bridge of the "stor" zone
// (blocking error) and a local zone absent from the reference (kept, no prune)
@@ -84,21 +98,72 @@ func Start(ctx context.Context, st *store.Store) (*Clusters, error) {
"dhcp-range": []any{"start-address=10.1.0.100,end-address=10.1.0.200"}}
b.Pending.Subnets["vlanz-10.2.0.0-24"] = map[string]any{"vnet": "v200", "gateway": "10.2.0.1"}
b.Commit()
b.Groups["admins"] = map[string]any{"comment": "Administrateurs"}
b.Groups["ops"] = map[string]any{"comment": "Exploitation"}
b.Roles["ConfSyncOps"] = "Sys.Audit,VM.Audit,VM.PowerMgmt"
b.Users["alice@pve"] = map[string]any{"enable": 1, "firstname": "Alice", "email": "alice@example.org", "groups": "admins"}
b.Users["bob@pve"] = map[string]any{"enable": 1, "firstname": "Bob", "email": "bob@example.org", "groups": "ops", "expire": 1893456000}
b.Shadow["alice"] = ref.Shadow["alice"]
b.Shadow["bob"] = ref.Shadow["bob"]
// nodes of the node replication: pve1 (master) and pve2 in pve-rennes,
// pve-a1 in site-a with other NIC names
n1, n2, a1 := ref.AddNode("pve1"), ref.AddNode("pve2"), a.AddNode("pve-a1")
n1.Net = map[string]map[string]any{
"eno1": {"type": "eth", "mtu": "9000"},
"eno2": {"type": "eth", "mtu": "9000"},
"bond0": {"type": "bond", "slaves": "eno1 eno2", "bond_mode": "802.3ad", "bond_xmit_hash_policy": "layer3+4", "mtu": "9000"},
"vmbr0": {"type": "bridge", "bridge_ports": "bond0", "bridge_vlan_aware": "1", "cidr": "10.0.0.11/24", "gateway": "10.0.0.1", "autostart": "1"},
"bond0.50": {"type": "vlan", "mtu": "9000"},
"vmbr1": {"type": "bridge", "bridge_ports": "bond0.50", "cidr": "10.5.0.11/24", "comments": "stockage iSCSI", "autostart": "1"},
}
n2.Net = map[string]map[string]any{
"eno1": {"type": "eth", "mtu": "9000"},
"eno2": {"type": "eth", "mtu": "9000"},
"bond0": {"type": "bond", "slaves": "eno1 eno2", "bond_mode": "active-backup", "mtu": "9000"},
"vmbr0": {"type": "bridge", "bridge_ports": "bond0", "cidr": "10.0.0.12/24", "gateway": "10.0.0.1", "autostart": "1"},
}
a1.Net = map[string]map[string]any{
"enp65s0f0": {"type": "eth"},
"enp65s0f1": {"type": "eth"},
"bond0": {"type": "bond", "slaves": "enp65s0f0 enp65s0f1", "bond_mode": "802.3ad", "bond_xmit_hash_policy": "layer3+4"},
"vmbr0": {"type": "bridge", "bridge_ports": "bond0", "bridge_vlan_aware": "1", "cidr": "192.168.10.21/24", "gateway": "192.168.10.1", "autostart": "1"},
}
n1.DNS = map[string]string{"search": "rennes.lab", "dns1": "10.0.0.53", "dns2": "10.0.0.54"}
n2.DNS = map[string]string{"search": "rennes.lab", "dns1": "10.0.0.53"}
a1.DNS = map[string]string{"search": "rennes.lab", "dns1": "10.0.0.53", "dns2": "10.0.0.54"}
mp := "defaults {\n user_friendly_names yes\n find_multipaths no\n}\nblacklist {\n devnode \"^sda\"\n}\ndevices {\n device {\n vendor \"NETAPP\"\n product \"LUN.*\"\n path_grouping_policy group_by_prio\n no_path_retry queue\n }\n}\n"
n1.Files["/etc/multipath.conf"] = pvetest.FakeFile{Content: mp}
n2.Files["/etc/multipath.conf"] = pvetest.FakeFile{Content: strings.Replace(mp, "no_path_retry queue", "no_path_retry 12", 1)}
n1.Files["/etc/multipath/wwids"] = pvetest.FakeFile{Content: "# Multipath wwids, Version : 1.0\n/3600a098038303053453f463045747a41/\n/3600a098038303053453f463045747a42/\n"}
n2.Files["/etc/multipath/wwids"] = pvetest.FakeFile{Content: "# Multipath wwids, Version : 1.0\n/3600a098038303053453f463045747a41/\n"}
n1.Files["/etc/chrony/chrony.conf"] = pvetest.FakeFile{Content: "server ntp1.rennes.lab iburst\nserver ntp2.rennes.lab iburst\ndriftfile /var/lib/chrony/chrony.drift\nmakestep 1.0 3\n"}
n2.Files["/etc/chrony/chrony.conf"] = pvetest.FakeFile{Content: "pool 2.debian.pool.ntp.org iburst\ndriftfile /var/lib/chrony/chrony.drift\nmakestep 1.0 3\n"}
a1.Files["/etc/chrony/chrony.conf"] = n1.Files["/etc/chrony/chrony.conf"]
n1.Files["/etc/sysctl.d/90-network.conf"] = pvetest.FakeFile{Content: "net.core.rmem_max = 16777216\nnet.core.wmem_max = 16777216\n"}
a1.Files["/etc/sysctl.d/90-network.conf"] = n1.Files["/etc/sysctl.d/90-network.conf"]
a1.Files["/etc/sysctl.d/99-local.conf"] = pvetest.FakeFile{Content: "vm.swappiness = 10\n"}
a1.Files["/etc/multipath.conf"] = n1.Files["/etc/multipath.conf"]
// legacy: PVE 8, refused by the version check
old := pvetest.New("8.4.1")
c := &Clusters{servers: []*pvetest.Server{ref, a, b, old}}
save := func(s *pvetest.Server, name, token string, refer bool, pos int,
save := func(s *pvetest.Server, name, role string, pos int,
mapping map[model.Kind]map[string]map[string]any, pol map[model.Kind]map[string]adapter.Policy) error {
sum := sha256.Sum256(s.Certificate().Raw)
return st.SaveCluster(ctx, "", store.Cluster{Name: name, Reference: refer, Position: pos, URL: s.URL,
TokenID: token, Secret: "demo-" + name, Fingerprint: hex.EncodeToString(sum[:]),
Mapping: mapping, Policies: pol})
host, port := s.SSHAddr()
err := st.SaveCluster(ctx, "", store.Cluster{Name: name, Role: role, Position: pos, URL: s.URL,
User: "root@pam", Password: pvetest.Password, TLS: store.TLSPin, Fingerprint: pve.FormatFingerprint(hex.EncodeToString(sum[:])),
SSHHost: host, SSHPort: port, Mapping: mapping, Policies: pol})
if err != nil {
return err
}
return st.Pin(ctx, name, "", s.SSHHostKey(), nil)
}
steps := []error{
save(ref, "ref-rennes", "confsync@pve!ro", true, 0, nil, nil),
save(a, "site-a", "confsync@pve!rw", false, 1, map[model.Kind]map[string]map[string]any{
save(ref, "pve-rennes", store.ClusterPrimary, 0, nil, nil),
save(a, "site-a", store.ClusterTarget, 1, map[model.Kind]map[string]map[string]any{
model.KindDatacenter: {"datacenter": {"next-id": "lower=2000,upper=2999"}},
model.KindZone: {
"vlanz": {"bridge": "vmbr1"},
@@ -106,16 +171,34 @@ func Start(ctx context.Context, st *store.Store) (*Clusters, error) {
"vxz": {"peers": "192.168.10.1,192.168.10.2"},
},
}, nil),
save(b, "site-b", "confsync@pve!rw", false, 2, map[model.Kind]map[string]map[string]any{
save(b, "site-b", store.ClusterTarget, 2, map[model.Kind]map[string]map[string]any{
model.KindZone: {
"vlanz": {"bridge": "vmbr0"},
"vxz": {"peers": "172.16.0.1,172.16.0.2"},
},
}, map[model.Kind]map[string]adapter.Policy{model.KindZone: {"mtu": adapter.Ignore}}),
save(old, "legacy-pve8", "confsync@pve!rw", false, 3, nil, nil),
save(old, "legacy-pve8", store.ClusterTarget, 3, nil, nil),
st.SaveSettings(ctx, store.Settings{Kinds: model.Kinds,
Policies: map[model.Kind]map[string]adapter.Policy{model.KindDatacenter: {"keyboard": adapter.Sync}}}),
}
node := func(cluster string, n *pvetest.Node) error {
host, port := n.SSHAddr()
if err := st.SaveNode(ctx, cluster, n.Name, host, port, "", false); err != nil {
return err
}
return st.PinNode(ctx, cluster, n.Name, n.SSHHostKey(), nil)
}
steps = append(steps,
st.SetNodes(ctx, "pve-rennes", []store.Discovered{{Name: "pve1", IP: "127.0.0.1"}, {Name: "pve2", IP: "127.0.0.1"}}),
st.SetNodes(ctx, "site-a", []store.Discovered{{Name: "pve-a1", IP: "127.0.0.1"}}),
node("pve-rennes", n1), node("pve-rennes", n2), node("site-a", a1),
st.SaveProfile(ctx, "", nodesync.Profile{Name: "réseau et stockage", Master: "pve-rennes/pve1",
Targets: []string{"pve-rennes/pve2", "site-a/pve-a1"},
Items: []string{"network", "dns", "multipath", "wwids", "chrony", "sysctl"},
Mapping: map[string]nodesync.TargetMap{
"pve-rennes/pve2": {Values: map[string]map[string]string{"network/vmbr1": {"cidr": "10.5.0.12/24"}}},
"site-a/pve-a1": {NICs: map[string]string{"eno1": "enp65s0f0", "eno2": "enp65s0f1"}},
}}))
for _, u := range []struct{ name, role string }{{"admin", store.RoleAdmin}, {"operateur", store.RoleOperator}, {"lecteur", store.RoleViewer}} {
steps = append(steps, st.SaveUser(ctx, u.name, u.role, Password))
}
internal/engine/access_test.go
+137
View File
@@ -0,0 +1,137 @@
package engine
import (
"context"
"encoding/json"
"strings"
"testing"
"pve-confsync/internal/adapter"
"pve-confsync/internal/config"
"pve-confsync/internal/model"
"pve-confsync/internal/pve/pvetest"
)
const aliceHash = "$5$salt$aliceHashValue"
func accessRef(t *testing.T) *pvetest.Server {
ref := pvetest.New("9.1.4")
t.Cleanup(ref.Close)
ref.Groups["admins"] = map[string]any{"comment": "Admins"}
ref.Groups["ops"] = map[string]any{"comment": "Ops"}
ref.Roles["Ops"] = "VM.PowerMgmt,Sys.Audit"
ref.Users["alice@pve"] = map[string]any{"enable": 1, "email": "alice@example.org", "groups": "ops,admins"}
ref.Users["bob@pve"] = map[string]any{"enable": 1, "groups": "ops", "expire": 0}
ref.Users["carol@pam"] = map[string]any{"enable": 1}
ref.Shadow["alice"] = aliceHash
ref.Shadow["bob"] = "$5$salt$bobHash"
ref.Shadow["orphan"] = "$5$salt$orphan" // no account: left aside
return ref
}
var accessKinds = []model.Kind{model.KindGroup, model.KindRole, model.KindUser, model.KindPassword}
func TestAccountsAndPasswords(t *testing.T) {
ctx := context.Background()
ref := accessRef(t)
cli := pvetest.New("9.0.10")
defer cli.Close()
cli.Users["alice@pve"] = map[string]any{"enable": 1, "email": "old@example.org", "firstname": "Al"}
cli.Shadow["alice"] = "$5$salt$oldHash"
cfg := &config.Config{Kinds: accessKinds, Reference: config.Cluster{Name: "ref"}, Clients: []config.Client{{Cluster: config.Cluster{Name: "site"}}}}
if err := cfg.Validate(); err != nil {
t.Fatal(err)
}
cl := &cfg.Clients[0]
src, err := Read(ctx, client(t, ref, "ref"), cfg.Kinds, adapter.ReadRunning)
if err != nil {
t.Fatal(err)
}
for _, o := range src.Objects {
if o.ID == "carol@pam" || o.ID == "orphan@pve" || o.ID == "Administrator" || o.ID == "root@pam" {
t.Fatalf("%s must not be read", o.Key())
}
}
cc := client(t, cli, "site")
cur, err := Read(ctx, cc, cfg.Kinds, adapter.ReadCurrent)
if err != nil {
t.Fatal(err)
}
plan := MakePlan(cfg, cl, src.Objects, cur.Objects)
if len(plan.Errors) > 0 {
t.Fatal(plan.Errors)
}
// 2 groups + 1 role + bob created, alice updated + 2 passwords
if len(plan.Changes) != 7 {
t.Fatalf("changes = %+v", plan.Changes)
}
b, _ := json.Marshal(plan)
if strings.Contains(string(b), "aliceHash") || strings.Contains(string(b), "oldHash") {
t.Fatalf("password hash in the plan JSON: %s", b)
}
res := Apply(ctx, cc, plan)
if res.Err != nil {
t.Fatalf("apply: %v (%+v)", res.Err, res.Steps)
}
if cli.Shadow["alice"] != aliceHash || cli.Shadow["bob"] != "$5$salt$bobHash" {
t.Fatalf("shadow = %v", cli.Shadow)
}
a := cli.Users["alice@pve"]
if a["email"] != "alice@example.org" || a["groups"] != "admins,ops" || a["firstname"] != nil {
t.Fatalf("alice = %v", a)
}
if cli.Roles["Ops"] != "Sys.Audit,VM.PowerMgmt" {
t.Fatalf("roles = %v", cli.Roles)
}
replan := func(actual []model.Object) Plan { return MakePlan(cfg, cl, src.Objects, actual) }
if err := Verify(ctx, cc, replan, cfg.Kinds); err != nil {
t.Fatalf("verify: %v", err)
}
}
func TestPasswordSelectionPullsAccountAndGroups(t *testing.T) {
ctx := context.Background()
ref := accessRef(t)
cli := pvetest.New("9.1.0")
defer cli.Close()
cfg := &config.Config{Kinds: accessKinds, Clients: []config.Client{{Cluster: config.Cluster{Name: "site"}}}}
src, _ := Read(ctx, client(t, ref, "ref"), cfg.Kinds, adapter.ReadRunning)
cur, _ := Read(ctx, client(t, cli, "site"), cfg.Kinds, adapter.ReadCurrent)
p := Filter(MakePlan(cfg, &cfg.Clients[0], src.Objects, cur.Objects), []string{"pve-password/bob@pve"})
var keys []string
for _, c := range p.Changes {
keys = append(keys, c.Key())
}
if got := strings.Join(keys, " "); got != "pve-group/ops pve-user/bob@pve pve-password/bob@pve" {
t.Fatalf("selection = %s", got)
}
}
func TestMissingParentIsAnError(t *testing.T) {
ctx := context.Background()
ref := accessRef(t)
cli := pvetest.New("9.1.0")
defer cli.Close()
// groups excluded from the selection: alice cannot be created in them
cfg := &config.Config{Kinds: accessKinds, Clients: []config.Client{{Cluster: config.Cluster{Name: "site"}}},
Objects: map[model.Kind]config.Selection{model.KindGroup: {Exclude: []string{"*"}}}}
src, _ := Read(ctx, client(t, ref, "ref"), cfg.Kinds, adapter.ReadRunning)
cur, _ := Read(ctx, client(t, cli, "site"), cfg.Kinds, adapter.ReadCurrent)
plan := MakePlan(cfg, &cfg.Clients[0], src.Objects, cur.Objects)
if len(plan.Errors) != 3 || !strings.Contains(strings.Join(plan.Errors, "\n"), "pve-user/alice@pve: pve-group/admins absent") {
t.Fatalf("errors = %v", plan.Errors)
}
}
func TestPasswordsNeedSSH(t *testing.T) {
s := pvetest.New("9.1.0")
defer s.Close()
c, err := config.Cluster{Name: "c", URL: s.URL, Password: pvetest.Password, Insecure: true}.PVE()
if err != nil {
t.Fatal(err)
}
if _, err := Read(context.Background(), c, accessKinds, adapter.ReadCurrent); err == nil || !strings.Contains(err.Error(), "SSH") {
t.Fatalf("err = %v", err)
}
}
internal/engine/apply.go
+1 -1
View File
@@ -85,7 +85,7 @@ func ApplyWithLog(ctx context.Context, c *pve.Client, plan Plan, notify func(Ste
err := c.Post(ctx, "cluster/sdn/lock", nil, &token)
res.log("sdn lock", err)
if err != nil {
res.Err = fmt.Errorf("SDN lock (pending changes on the client?): %w", err)
res.Err = fmt.Errorf("SDN lock (pending changes on the target?): %w", err)
return res
}
rollback := func(cause error) {
internal/engine/engine.go
+80 -13
View File
@@ -4,6 +4,7 @@ package engine
import (
"context"
"encoding/json"
"fmt"
"sort"
"time"
@@ -42,6 +43,9 @@ func Read(ctx context.Context, c *pve.Client, kinds []model.Kind, mode adapter.R
if need[model.KindVNet] {
need[model.KindZone] = true
}
if need[model.KindPassword] {
need[model.KindUser] = true
}
var all []model.Object
for _, a := range adapter.All() {
if !need[a.Kind()] {
@@ -74,21 +78,43 @@ const (
// Change is one planned action. Set holds the new values, Old the current ones.
type Change struct {
Kind model.Kind `json:"kind" yaml:"kind"`
ID string `json:"id" yaml:"id"`
Op Op `json:"op" yaml:"op"`
Set map[string]any `json:"set,omitempty" yaml:"set,omitempty"`
Unset []string `json:"unset,omitempty" yaml:"unset,omitempty"`
Old map[string]any `json:"old,omitempty" yaml:"old,omitempty"`
Target model.Object `json:"-" yaml:"-"`
Kind model.Kind `json:"kind"`
ID string `json:"id"`
Op Op `json:"op"`
Set map[string]any `json:"set,omitempty"`
Unset []string `json:"unset,omitempty"`
Old map[string]any `json:"old,omitempty"`
Target model.Object `json:"-"`
}
// MarshalJSON hides the secret fields (password hashes): plans are shown in the
// UI and kept in the journal, the real values only live in memory until applied.
func (c Change) MarshalJSON() ([]byte, error) {
type plain Change
red := func(m map[string]any) map[string]any {
if m == nil {
return nil
}
out := make(map[string]any, len(m))
for k, v := range m {
if adapter.IsSensitive(c.Kind, k) {
v = adapter.Redact(v)
}
out[k] = v
}
return out
}
p := plain(c)
p.Set, p.Old = red(c.Set), red(c.Old)
return json.Marshal(p)
}
// Plan is the list of changes for one client, in execution order.
type Plan struct {
Client string `json:"client" yaml:"client"`
Changes []Change `json:"changes" yaml:"changes"`
Errors []string `json:"errors,omitempty" yaml:"errors,omitempty"`
Warnings []string `json:"warnings,omitempty" yaml:"warnings,omitempty"`
Client string `json:"client"`
Changes []Change `json:"changes"`
Errors []string `json:"errors,omitempty"`
Warnings []string `json:"warnings,omitempty"`
}
// HasSDN tells whether the plan writes SDN objects (lock + reload needed).
@@ -132,13 +158,13 @@ func MakePlan(cfg *config.Config, cl *config.Client, ref, actual []model.Object)
o, exists := cur[r.ID]
if !exists {
if a.Singleton() {
plan.Errors = append(plan.Errors, fmt.Sprintf("%s: not found on client", r.Key()))
plan.Errors = append(plan.Errors, fmt.Sprintf("%s: not found on the target", r.Key()))
continue
}
for _, f := range a.Required(model.Object{Kind: k, ID: r.ID, Fields: desired}) {
if _, ok := desired[f]; !ok {
plan.Errors = append(plan.Errors,
fmt.Sprintf("%s: field %q required for creation, add it to the mapping of client %s", r.Key(), f, cl.Name))
fmt.Sprintf("%s: field %q required for creation, add it to the mapping of target %s", r.Key(), f, cl.Name))
}
}
plan.Changes = append(plan.Changes, Change{Kind: k, ID: r.ID, Op: OpCreate, Set: desired,
@@ -187,6 +213,7 @@ func MakePlan(cfg *config.Config, cl *config.Client, ref, actual []model.Object)
}
}
}
checkParents(&plan, cl.Name, actual)
// deletions last, dependents first (subnets, vnets, zones)
sort.SliceStable(deletes, func(i, j int) bool {
if deletes[i].Kind != deletes[j].Kind {
@@ -198,6 +225,46 @@ func MakePlan(cfg *config.Config, cl *config.Client, ref, actual []model.Object)
return plan
}
// checkParents blocks the objects whose parents (groups of an account, account
// of a password) are neither on the target nor created by the plan.
func checkParents(plan *Plan, client string, actual []model.Object) {
have := map[string]bool{}
for _, o := range actual {
have[o.Key()] = true
}
for _, c := range plan.Changes {
if c.Op == OpCreate {
have[c.Key()] = true
}
}
for _, c := range plan.Changes {
if c.Op == OpDelete {
continue
}
for _, p := range adapter.Parents(Desired(c)) {
if !have[p] {
plan.Errors = append(plan.Errors, fmt.Sprintf("%s: %s absent on target %s and not replicated", c.Key(), p, client))
}
}
}
}
// Desired is the object as it will be after the change (fields of the target
// with the new values).
func Desired(c Change) model.Object {
f := map[string]any{}
for k, v := range c.Target.Fields {
f[k] = v
}
for _, k := range c.Unset {
delete(f, k)
}
for k, v := range c.Set {
f[k] = v
}
return model.Object{Kind: c.Kind, ID: c.ID, Fields: f}
}
func kindRank(k model.Kind) int {
for i, x := range model.Kinds {
if x == k {
internal/engine/engine_test.go
+9 -3
View File
@@ -14,17 +14,23 @@ import (
"pve-confsync/internal/pve/pvetest"
)
// client connects to a fake cluster like the web UI does: API as root@pam,
// pinned certificate, SSH with the pinned host key.
func client(t *testing.T, s *pvetest.Server, name string) *pve.Client {
t.Helper()
sum := sha256.Sum256(s.Certificate().Raw)
c, err := pve.New(pve.Options{Name: name, BaseURL: s.URL, TokenID: "confsync@pve!t", TokenSecret: "secret",
Fingerprint: hex.EncodeToString(sum[:])})
host, port := s.SSHAddr()
c, err := config.Cluster{Name: name, URL: s.URL, User: "root@pam", Password: pvetest.Password,
Fingerprint: hex.EncodeToString(sum[:]), SSHHost: host, SSHPort: port, SSHHostKey: s.SSHHostKey()}.PVE()
if err != nil {
t.Fatal(err)
}
return c
}
// sdnKinds: the kinds of the SDN and datacenter tests.
var sdnKinds = []model.Kind{model.KindDatacenter, model.KindZone, model.KindVNet, model.KindSubnet}
// reference: two zones (vlan, vxlan), one unsupported (evpn), vnets, subnets, tags.
func reference(t *testing.T) *pvetest.Server {
ref := pvetest.New("9.1.4")
@@ -49,7 +55,7 @@ func reference(t *testing.T) *pvetest.Server {
func testConfig() *config.Config {
return &config.Config{
Kinds: model.Kinds,
Kinds: sdnKinds,
Clients: []config.Client{{
Cluster: config.Cluster{Name: "site-a"},
Mapping: map[model.Kind]map[string]map[string]any{
internal/engine/select.go
+43
View File
@@ -7,6 +7,7 @@ import (
"sort"
"strings"
"pve-confsync/internal/adapter"
"pve-confsync/internal/model"
)
@@ -50,6 +51,11 @@ func Filter(p Plan, keys []string) Plan {
if v, ok := c.Set["vnet"].(string); ok && byKey[string(model.KindVNet)+"/"+v].Op == OpCreate {
add(string(model.KindVNet) + "/" + v)
}
for _, p := range adapter.Parents(Desired(c)) {
if byKey[p].Op == OpCreate {
add(p)
}
}
case OpDelete:
// children deleted in the same plan must go first
for _, d := range p.Changes {
@@ -79,6 +85,9 @@ func Filter(p Plan, keys []string) Plan {
if v, ok := c.Set["vnet"].(string); ok {
blocking[string(model.KindVNet)+"/"+v] = true
}
for _, p := range adapter.Parents(Desired(c)) {
blocking[p] = true
}
}
out := Plan{Client: p.Client, Warnings: p.Warnings}
// errors about other objects ("kind/id: ...") do not block this selection
@@ -142,3 +151,37 @@ func (p Plan) Digest() string {
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:12])
}
// Skip removes from a plan the changes of the kinds left out of a replication
// ("not synchronized" in the dry-run), then keeps what the remaining changes
// need as Filter does: a group to create for an account stays in the plan.
// keys narrows the plan to some objects first (empty: every object).
func Skip(p Plan, keys []string, skip []model.Kind) Plan {
if len(skip) == 0 {
return Filter(p, keys)
}
out := map[model.Kind]bool{}
for _, k := range skip {
out[k] = true
}
if len(keys) == 0 {
for _, c := range p.Changes {
keys = append(keys, c.Key())
}
for _, e := range p.Errors {
if k, ok := ErrorKey(e); ok {
keys = append(keys, k)
}
}
}
var kept []string
for _, k := range keys {
if kind, _, _ := strings.Cut(k, "/"); !out[model.Kind(kind)] {
kept = append(kept, k)
}
}
if len(kept) == 0 {
return Plan{Client: p.Client, Warnings: p.Warnings}
}
return Filter(p, kept)
}
internal/engine/skip_test.go
+41
View File
@@ -0,0 +1,41 @@
package engine
import (
"reflect"
"testing"
"pve-confsync/internal/model"
)
// Items left out of a run ("do not replicate") leave the plan, except what the
// remaining changes need: the group to create for a new account.
func TestSkipKinds(t *testing.T) {
p := Plan{Client: "c", Changes: []Change{
{Kind: model.KindGroup, ID: "ops", Op: OpCreate, Set: map[string]any{"comment": "Ops"}},
{Kind: model.KindGroup, ID: "dev", Op: OpUpdate, Set: map[string]any{"comment": "Dev"}},
{Kind: model.KindUser, ID: "bob@pve", Op: OpCreate, Set: map[string]any{"groups": "ops"}},
{Kind: model.KindPassword, ID: "bob@pve", Op: OpCreate, Set: map[string]any{"hash": "$5$x"}},
{Kind: model.KindVNet, ID: "v1", Op: OpUpdate, Set: map[string]any{"alias": "a"}},
}, Errors: []string{"sdn-zone/z1: field \"bridge\" required for creation, add it to the mapping of target c"}}
keys := func(p Plan) []string {
var out []string
for _, c := range p.Changes {
out = append(out, c.Key())
}
return out
}
got := Skip(p, nil, []model.Kind{model.KindGroup, model.KindZone, model.KindVNet})
if want := []string{"pve-group/ops", "pve-user/bob@pve", "pve-password/bob@pve"}; !reflect.DeepEqual(keys(got), want) {
t.Fatalf("changes: %v, want %v", keys(got), want)
}
if len(got.Errors) != 0 {
t.Fatalf("the error of a skipped kind must not block: %v", got.Errors)
}
if got := Skip(p, nil, model.Kinds); len(got.Changes) != 0 || len(got.Errors) != 0 {
t.Fatalf("everything skipped: %+v", got)
}
if got := Skip(p, nil, nil); len(got.Changes) != len(p.Changes) || len(got.Errors) != 1 {
t.Fatalf("nothing skipped: %+v", got)
}
}
internal/model/model.go
+16 -12
View File
@@ -1,6 +1,6 @@
// Package model holds the generic, normalized representation of synchronized objects.
// Field values are canonical strings or sorted string lists, so that two clusters
// can be compared field by field and the result serialized to YAML without loss.
// can be compared field by field.
package model
import (
@@ -15,32 +15,36 @@ type Kind string
const (
KindDatacenter Kind = "datacenter-options"
KindGroup Kind = "pve-group" // groups of user.cfg
KindRole Kind = "pve-role" // custom roles of user.cfg
KindUser Kind = "pve-user" // accounts of the pve realm
KindPassword Kind = "pve-password" // their password hashes (SSH, priv/shadow.cfg)
KindZone Kind = "sdn-zone"
KindVNet Kind = "sdn-vnet"
KindSubnet Kind = "sdn-subnet"
)
// Kinds lists the kinds in dependency order (creation order).
var Kinds = []Kind{KindDatacenter, KindZone, KindVNet, KindSubnet}
var Kinds = []Kind{KindDatacenter, KindGroup, KindRole, KindUser, KindPassword, KindZone, KindVNet, KindSubnet}
// Object is one configuration object of a cluster.
// Field values are either string or []string (see Normalize).
type Object struct {
Kind Kind `yaml:"kind" json:"kind"`
ID string `yaml:"id" json:"id"`
Fields map[string]any `yaml:"fields" json:"fields"`
Kind Kind `json:"kind"`
ID string `json:"id"`
Fields map[string]any `json:"fields"`
}
// Key is the unique identity of an object inside a cluster.
func (o Object) Key() string { return string(o.Kind) + "/" + o.ID }
// Snapshot is the exported state of one cluster (YAML, versionable in Git).
// Snapshot is the state read from one cluster.
type Snapshot struct {
Cluster string `yaml:"cluster"`
PVEVersion string `yaml:"pve_version,omitempty"`
ExportedAt time.Time `yaml:"exported_at"`
Objects []Object `yaml:"objects"`
Warnings []string `yaml:"warnings,omitempty"`
Cluster string `json:"cluster"`
PVEVersion string `json:"pveVersion,omitempty"`
ExportedAt time.Time `json:"readAt"`
Objects []Object `json:"objects"`
Warnings []string `json:"warnings,omitempty"`
}
// Sort orders objects by kind (dependency order) then by id.
@@ -57,7 +61,7 @@ func Sort(objs []Object) {
})
}
// Normalize turns a YAML/JSON decoded field map into canonical values:
// Normalize turns a JSON decoded field map into canonical values:
// scalars become strings, lists become sorted []string.
func Normalize(fields map[string]any) map[string]any {
out := make(map[string]any, len(fields))
internal/nodesync/analyze.go
+341
View File
@@ -0,0 +1,341 @@
package nodesync
import (
"context"
"errors"
"fmt"
"sort"
"sync"
"time"
"pve-confsync/internal/model"
)
// NodeInfo is what was read from one node.
type NodeInfo struct {
Key string `json:"name"` // "cluster/node", "name" like the cluster cards
Cluster string `json:"cluster"`
Node string `json:"node"`
Version string `json:"version,omitempty"`
Error string `json:"error,omitempty"`
Warnings []string `json:"warnings,omitempty"`
ReadAt time.Time `json:"readAt"`
Millis int64 `json:"ms"`
}
// Field compares one field of an interface or of the DNS servers.
type Field struct {
Name string `json:"name"`
Policy Policy `json:"policy"`
Ref any `json:"ref,omitempty"` // master
Mapping any `json:"mapping,omitempty"` // value of the target mapping
Current any `json:"current,omitempty"` // target
Target any `json:"target,omitempty"` // after the replication
Change string `json:"change,omitempty"` // add, set, unset or empty
}
// Row statuses (same as the cluster rows).
const (
StatusSame = "same"
StatusCreate = "create"
StatusUpdate = "update"
StatusDelete = "delete"
StatusExtra = "extra" // only on the target, kept
StatusError = "error"
)
// Row is one object of a target: an interface, the DNS servers or a file.
type Row struct {
Kind string `json:"kind"` // item id
ID string `json:"id"`
Key string `json:"key"`
Status string `json:"status"`
Fields []Field `json:"fields,omitempty"`
Diff []Line `json:"diff,omitempty"`
Errors []string `json:"errors,omitempty"`
}
// Target is the state of one target node.
type Target struct {
NodeInfo
Plan Plan `json:"plan"`
Digest string `json:"digest"`
Rows []Row `json:"rows"`
Counts map[string]int `json:"counts"`
}
// State is a full comparison of the master with its targets.
type State struct {
At time.Time `json:"at"`
Profile string `json:"profile"`
Items []string `json:"items"`
Prune bool `json:"prune"`
Master NodeInfo `json:"reference"` // "reference" like the cluster state
Targets []Target `json:"clients"`
prof *Profile
master Snapshot
targets map[string]Snapshot
eps map[string]Endpoint
}
// Analyze reads the master and the targets (in parallel) and compares them.
// only limits the targets (empty: all). A node that cannot be read is
// reported in its Error field, it does not stop the others.
func Analyze(ctx context.Context, prof *Profile, eps map[string]Endpoint, only []string) *State {
st := &State{At: time.Now().UTC(), Profile: prof.Name, Items: prof.Items, Prune: prof.Prune, Targets: []Target{},
prof: prof, targets: map[string]Snapshot{}, eps: eps}
st.Master, st.master = read(ctx, eps, prof.Master, prof.Items)
want := map[string]bool{}
for _, n := range only {
want[n] = true
}
var wg sync.WaitGroup
var mu sync.Mutex
for _, key := range prof.Targets {
if len(only) > 0 && !want[key] {
continue
}
st.Targets = append(st.Targets, Target{NodeInfo: info(key)})
idx := len(st.Targets) - 1
wg.Add(1)
go func() {
defer wg.Done()
ni, snap := read(ctx, eps, key, prof.Items)
mu.Lock()
defer mu.Unlock()
st.Targets[idx].NodeInfo = ni
if ni.Error == "" {
st.targets[key] = snap
}
}()
}
wg.Wait()
for i := range st.Targets {
t := &st.Targets[i]
t.Counts = map[string]int{}
if t.Error != "" || st.Master.Error != "" {
continue
}
t.Plan = MakePlan(prof, st.master, st.targets[t.Key])
t.Digest = t.Plan.Digest()
t.Rows = rows(prof, st.master, st.targets[t.Key], t.Plan)
for _, r := range t.Rows {
t.Counts[r.Status]++
}
}
return st
}
func info(key string) NodeInfo {
ni := NodeInfo{Key: key}
for i := len(key) - 1; i >= 0; i-- {
if key[i] == '/' {
ni.Cluster, ni.Node = key[:i], key[i+1:]
break
}
}
return ni
}
func read(ctx context.Context, eps map[string]Endpoint, key string, items []string) (NodeInfo, Snapshot) {
ni := info(key)
ni.ReadAt = time.Now().UTC()
start := time.Now()
ep, ok := eps[key]
if !ok {
ni.Error = fmt.Sprintf("unknown node %s (discover the nodes of its cluster)", key)
return ni, Snapshot{}
}
snap, err := Read(ctx, ep, items)
if err != nil {
ni.Error = err.Error()
}
ni.Version = snap.Version
ni.Warnings = snap.Warnings
ni.Millis = time.Since(start).Milliseconds()
return ni, snap
}
// rows describes every object of the master and of the target.
func rows(prof *Profile, master, target Snapshot, plan Plan) []Row {
changes := map[string]Change{}
for _, c := range plan.Changes {
changes[c.Key()] = c
}
errs := map[string][]string{}
for _, e := range plan.Errors {
if e.Key != "" {
errs[e.Key] = append(errs[e.Key], e.Message)
}
}
tm := prof.Mapping[target.Key]
var out []Row
seen := map[string]bool{}
status := func(key string, inMaster bool) string {
ch, has := changes[key]
switch {
case len(errs[key]) > 0:
return StatusError
case has:
return string(ch.Op)
case !inMaster:
return StatusExtra
}
return StatusSame
}
addAPI := func(item, id string, ref, cur map[string]any) {
key := item + "/" + id
seen[key] = true
ch := changes[key]
r := Row{Kind: item, ID: id, Key: key, Status: status(key, ref != nil), Errors: errs[key]}
names := map[string]bool{}
for _, m := range []map[string]any{ref, cur, ch.Set} {
for f := range m {
names[f] = true
}
}
for f := range tm.Values[key] {
names[f] = true
}
list := make([]string, 0, len(names))
for f := range names {
list = append(list, f)
}
sort.Strings(list)
unset := map[string]bool{}
for _, f := range ch.Unset {
unset[f] = true
}
for _, f := range list {
fd := Field{Name: f, Policy: prof.Policy(item, f), Ref: ref[f], Current: cur[f]}
if v, ok := tm.Values[key][f]; ok {
fd.Mapping = v
}
switch {
case ch.Op == OpDelete || unset[f]:
fd.Change = "unset"
case ch.Set[f] != nil:
fd.Target = ch.Set[f]
fd.Change = "set"
if fd.Current == nil {
fd.Change = "add"
}
case fd.Current != nil:
fd.Target = fd.Current
}
r.Fields = append(r.Fields, fd)
}
out = append(out, r)
}
for _, id := range prof.Items {
it, _ := ItemByID(id)
switch it.Kind {
case KindNetwork:
cur := map[string]model.Object{}
for _, o := range target.Network {
cur[o.ID] = o
}
for _, m := range master.Network {
tid := rename(m.ID, tm.NICs)
addAPI("network", tid, m.Fields, cur[tid].Fields)
}
for _, o := range target.Network {
if !seen["network/"+o.ID] {
addAPI("network", o.ID, nil, o.Fields)
}
}
case KindDNS:
addAPI("dns", "dns", master.DNS, target.DNS)
case KindFile:
paths := map[string]bool{}
for p := range master.Files[id] {
paths[p] = true
}
for p := range target.Files[id] {
paths[p] = true
}
if len(master.Files[id]) == 0 && !it.Glob {
continue
}
for p := range paths {
key := id + "/" + p
_, inMaster := master.Files[id][p]
r := Row{Kind: id, ID: p, Key: key, Status: status(key, inMaster), Errors: errs[key], Diff: changes[key].Diff}
out = append(out, r)
}
}
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Kind != out[j].Kind {
return itemRank(out[i].Kind) < itemRank(out[j].Kind)
}
return out[i].ID < out[j].ID
})
return out
}
// TargetPlan is the plan of one target restricted to a selection of objects.
type TargetPlan struct {
Client string `json:"client"` // "client" like the cluster plans
Plan Plan `json:"plan"`
Digest string `json:"digest"`
Network bool `json:"network"`
Reloads []string `json:"reloads,omitempty"`
Error string `json:"error,omitempty"`
}
// Plans returns the plans of the targets restricted to the given objects
// (empty: everything), as a dry-run shows them and an apply checks them.
func (st *State) Plans(keys []string) []TargetPlan {
var out []TargetPlan
for _, t := range st.Targets {
tp := TargetPlan{Client: t.Key}
switch {
case st.Master.Error != "":
tp.Error = "master: " + st.Master.Error
case t.Error != "":
tp.Error = t.Error
default:
tp.Plan = Filter(t.Plan, keys)
tp.Digest = tp.Plan.Digest()
tp.Network = tp.Plan.Network()
tp.Reloads = tp.Plan.Reloads()
}
out = append(out, tp)
}
return out
}
// Apply writes one plan on its target, then reads the target again and checks
// that the selected objects are in line with the master.
func (st *State) Apply(ctx context.Context, tp TargetPlan, keys []string, log func(Step)) error {
ep, ok := st.eps[tp.Client]
if !ok {
return fmt.Errorf("unknown node %q", tp.Client)
}
if err := Apply(ctx, ep, tp.Plan, log); err != nil {
return err
}
start := time.Now()
err := st.verify(ctx, ep, keys)
s := Step{At: start.UTC(), Action: "verify"}
if err != nil {
s.Error = err.Error()
}
log(s)
return err
}
func (st *State) verify(ctx context.Context, ep Endpoint, keys []string) error {
snap, err := Read(ctx, ep, st.prof.Items)
if err != nil {
return err
}
p := Filter(MakePlan(st.prof, st.master, snap), keys)
var errs []error
for _, c := range p.Changes {
errs = append(errs, fmt.Errorf("still different after apply: %s", c.Key()))
}
return errors.Join(errs...)
}
internal/nodesync/apply.go
+333
View File
@@ -0,0 +1,333 @@
package nodesync
import (
"bytes"
"context"
"encoding/base64"
"errors"
"fmt"
"net/url"
"sort"
"strings"
"time"
"pve-confsync/internal/pve"
)
// Step is the outcome of one action, for the execution journal.
type Step struct {
At time.Time `json:"at"`
Action string `json:"action"`
Error string `json:"error,omitempty"`
}
// Timing of an apply; variables for the tests.
var (
// RestoreDelay: when the network was reloaded, the node restores its
// previous configuration by itself after this delay unless pve-confsync
// reaches it again over SSH and cancels the restore.
RestoreDelay = 120 * time.Second
// ProbeTries and ProbeWait: attempts to reach the node after the reload.
ProbeTries = 6
ProbeWait = 5 * time.Second
TaskPoll = 2 * time.Second
)
// BackupRoot is where the previous files are kept on the node, one folder
// per apply, with the restore.sh script that puts them back.
const BackupRoot = "/var/lib/pve-confsync/backup"
// Apply writes a plan on a target node:
// 1. one SSH script checks that every file is still the one of the dry-run,
// backs up the files (and /etc/network/interfaces, /etc/resolv.conf when
// the API writes them), writes restore.sh, then writes the new files
// atomically (temporary file then mv, mode and owner kept);
// 2. the API writes the interfaces (in interfaces.new) and the DNS servers;
// 3. when the network is reloaded, a systemd timer is armed on the node: it
// runs restore.sh and ifreload -a after RestoreDelay;
// 4. the reload commands run (sysctl, chrony, multipath, network last);
// 5. the node must answer again over SSH and the API: the timer is stopped.
//
// A failure before the network reload restores the backups (and drops
// interfaces.new); a reload failure restores them and runs the reloads again.
func Apply(ctx context.Context, ep Endpoint, plan Plan, notify func(Step)) error {
log := func(action string, err error) error {
s := Step{At: time.Now().UTC(), Action: action}
if err != nil {
s.Error = err.Error()
}
if notify != nil {
notify(s)
}
return err
}
if len(plan.Errors) > 0 {
return log("plan check", fmt.Errorf("plan has %d error(s), nothing applied", len(plan.Errors)))
}
if plan.Empty() {
return nil
}
if ep.Shell == nil {
return log("ssh", fmt.Errorf("%s: SSH host key not recorded yet, test the node first", ep.Key))
}
if _, err := NodeVersion(ctx, ep); err != nil {
return log("version check", err)
}
stamp := time.Now().UTC().Format("20060102-150405")
dir := BackupRoot + "/" + stamp
unit := "pve-confsync-restore-" + stamp
var netAPI, dns bool
for _, c := range plan.Changes {
netAPI = netAPI || c.Item == "network"
dns = dns || c.Item == "dns"
}
if _, err := ep.Shell.Run(ctx, prepareScript(dir), prepareInput(plan, netAPI, dns)); err != nil {
return log("check, back up and write the files ("+dir+")", err)
}
log("check, back up and write the files ("+dir+")", nil)
restore := func(cause error) error {
_, rerr := ep.Shell.Run(ctx, restoreScript(dir), nil)
log("restore the previous files", rerr)
if rerr != nil {
return errors.Join(cause, fmt.Errorf("restore failed, run sh %s/restore.sh on the node: %w", dir, rerr))
}
return cause
}
for _, c := range plan.Changes {
if c.Item != "network" && c.Item != "dns" {
continue
}
if err := log(describe(ep, c), writeAPI(ctx, ep, c)); err != nil {
return restore(err)
}
}
network := plan.Network()
if network {
_, err := ep.Shell.Run(ctx, armScript(dir, unit), nil)
if err := log(fmt.Sprintf("arm the automatic restore (%s, %s)", unit, RestoreDelay), err); err != nil {
return restore(err)
}
}
var done []string // reload commands already run
for _, cmd := range plan.Reloads() {
var err error
action := cmd
if cmd == "ifreload -a" && netAPI {
action = "PUT /nodes/" + ep.Node + "/network (ifreload -a)"
err = reloadNetworkAPI(ctx, ep)
} else {
_, err = ep.Shell.Run(ctx, reloadScript(cmd), nil)
}
done = append(done, cmd)
if log("reload: "+action, err) == nil {
continue
}
cause := fmt.Errorf("reload %q: %w", cmd, err)
if network && !reachable(ctx, ep) {
return log("node unreachable", fmt.Errorf("%w; the node is unreachable: it restores its previous configuration by itself within %s", cause, RestoreDelay))
}
// back to the previous files, which the reloads put in effect again
err = restore(cause)
for _, c := range done {
_, rerr := ep.Shell.Run(ctx, reloadScript(c), nil)
log("reload after restore: "+c, rerr)
}
if network {
_, derr := ep.Shell.Run(ctx, disarmScript(unit), nil)
log("stop the automatic restore", derr)
}
return err
}
if network {
if !reachable(ctx, ep) {
return log("reach the node again", fmt.Errorf("%s unreachable after the network reload: it restores its previous configuration by itself within %s", ep.Key, RestoreDelay))
}
log("reach the node again (SSH and API)", nil)
_, err := ep.Shell.Run(ctx, disarmScript(unit), nil)
if err := log("stop the automatic restore", err); err != nil {
return fmt.Errorf("stop the automatic restore (systemctl stop %s.timer on the node): %w", unit, err)
}
}
return nil
}
func describe(ep Endpoint, c Change) string {
if c.Item == "dns" {
return "PUT /nodes/" + ep.Node + "/dns"
}
return fmt.Sprintf("%s %s/%s", c.Op, c.Item, c.ID)
}
// reachable opens a new SSH connection and calls the API, a few times.
func reachable(ctx context.Context, ep Endpoint) bool {
for i := 0; i < ProbeTries; i++ {
if i > 0 {
select {
case <-ctx.Done():
return false
case <-time.After(ProbeWait):
}
}
sh := ep.Shell
if ep.Dial != nil {
var err error
if sh, err = ep.Dial(); err != nil {
continue
}
}
if _, err := sh.Run(ctx, "# confsync:probe\ntrue", nil); err != nil {
continue
}
if _, err := NodeVersion(ctx, ep); err == nil {
return true
}
}
return false
}
func nodePath(ep Endpoint, rest string) string {
return "nodes/" + url.PathEscape(ep.Node) + "/" + rest
}
// writeAPI writes one interface (POST, PUT with every field, DELETE) or the
// DNS servers (PUT with every server).
func writeAPI(ctx context.Context, ep Endpoint, c Change) error {
if c.Item == "dns" {
f := url.Values{}
for k, v := range c.Full {
f.Set(k, pve.Str(v))
}
return ep.API.Put(ctx, nodePath(ep, "dns"), f, nil)
}
switch c.Op {
case OpCreate:
f := form(c.Full)
f.Set("iface", c.ID)
return ep.API.Post(ctx, nodePath(ep, "network"), f, nil)
case OpUpdate:
f := form(c.Full)
if len(c.Unset) > 0 {
f.Set("delete", strings.Join(c.Unset, ","))
}
return ep.API.Put(ctx, nodePath(ep, "network/"+url.PathEscape(c.ID)), f, nil)
case OpDelete:
return ep.API.Delete(ctx, nodePath(ep, "network/"+url.PathEscape(c.ID)), nil, nil)
}
return fmt.Errorf("unknown op %q", c.Op)
}
func form(fields map[string]any) url.Values {
f := url.Values{}
keys := make([]string, 0, len(fields))
for k := range fields {
keys = append(keys, k)
}
sort.Strings(keys)
for _, k := range keys {
f.Set(k, pve.Str(fields[k]))
}
return f
}
// reloadNetworkAPI applies interfaces.new (PUT /nodes/{node}/network: rename
// then ifreload -a in a task) and waits for the task.
func reloadNetworkAPI(ctx context.Context, ep Endpoint) error {
var upid string
if err := ep.API.Put(ctx, nodePath(ep, "network"), url.Values{}, &upid); err != nil {
return err
}
return ep.API.WaitTask(ctx, upid, TaskPoll)
}
// ---- scripts run on the node (the first line is a marker for the fake node
// of the tests) ----
// prepareInput is the plan given to prepareScript on its standard input:
// "W <mode> <owner> <hash> <path>" then the content in base64 on one line,
// "D - - <hash> <path>" (delete) or "B - - - <path>" (backup only).
func prepareInput(plan Plan, netAPI, dns bool) []byte {
var b bytes.Buffer
if netAPI {
b.WriteString("B - - - /etc/network/interfaces\nB - - - /etc/network/interfaces.new\n")
}
if dns {
b.WriteString("B - - - /etc/resolv.conf\n")
}
for _, c := range plan.Changes {
switch {
case c.Item == "network" || c.Item == "dns":
case c.Op == OpDelete:
fmt.Fprintf(&b, "D - - %s %s\n", c.Hash, c.ID)
default:
fmt.Fprintf(&b, "W %s %s %s %s\n%s\n", c.Mode, c.Owner, c.Hash, c.ID, base64.StdEncoding.EncodeToString(c.Content))
}
}
return b.Bytes()
}
func prepareScript(dir string) string { return prepareScriptIn(dir, "") }
// prepareScriptIn prefixes every path with root (empty on a node): the tests
// run the script for real in a temporary folder.
func prepareScriptIn(dir, root string) string {
return `# confsync:prepare ` + dir + `
set -u
R='` + root + `'
B="$R` + dir + `"
umask 077
mkdir -p "$B/files" || exit 1
cat > "$B/plan" || exit 1
sum() { if [ -e "$1" ]; then sha256sum < "$1" | cut -d' ' -f1; else echo absent; fi; }
# 1. every file must be the one read by the dry-run
while read -r op mode owner h p; do
case "$op" in W|D) ;; *) continue ;; esac
[ "$(sum "$R$p")" = "$h" ] || { echo "changed since the dry-run: $p" >&2; exit 3; }
done < "$B/plan"
# 2. backups and restore script
: > "$B/restore.sh"
while read -r op mode owner h p; do
case "$op" in W|D|B) ;; *) continue ;; esac
if [ -e "$R$p" ]; then
mkdir -p "$B/files$(dirname "$p")" && cp -p "$R$p" "$B/files$p" || exit 1
printf "cp -p '%s' '%s'\n" "$B/files$p" "$R$p" >> "$B/restore.sh"
else
printf "rm -f '%s'\n" "$R$p" >> "$B/restore.sh"
fi
done < "$B/plan"
# 3. writes: temporary file in the same folder, then mv
while read -r op mode owner h p; do
case "$op" in
W)
read -r data
mkdir -p "$(dirname "$R$p")" || exit 4
t=$(mktemp "$R$p.confsync.XXXXXX") || exit 4
if printf '%s' "$data" | base64 -d > "$t" && chmod "$mode" "$t" && chown "$owner" "$t" && mv -f "$t" "$R$p"; then :; else
rm -f "$t"; sh "$B/restore.sh"; echo "write failed: $p" >&2; exit 4
fi ;;
D)
rm -f "$R$p" || { sh "$B/restore.sh"; echo "delete failed: $p" >&2; exit 4; } ;;
esac
done < "$B/plan"
echo ok
`
}
func restoreScript(dir string) string {
return "# confsync:restore " + dir + "\nsh '" + dir + "/restore.sh'\n"
}
func armScript(dir, unit string) string {
return fmt.Sprintf("# confsync:arm %s\nsystemd-run --quiet --unit=%s --on-active=%ds --timer-property=AccuracySec=1s /bin/sh -c 'sh %s/restore.sh; ifreload -a'\n",
unit, unit, int(RestoreDelay.Seconds()), dir)
}
func disarmScript(unit string) string {
return "# confsync:disarm " + unit + "\nsystemctl stop " + unit + ".timer\n"
}
func reloadScript(cmd string) string {
return "# confsync:reload " + cmd + "\n" + cmd + "\n"
}
internal/nodesync/diff.go
+148
View File
@@ -0,0 +1,148 @@
package nodesync
import (
"strconv"
"strings"
)
// Line is one line of a unified diff: Op is ' ' (context), '-' (target only)
// or '+' (written by the replication). A hunk starts with Op '@' and Text
// "@@ -a,b +c,d @@".
type Line struct {
Op string `json:"op"`
Text string `json:"text"`
}
const diffContext = 3
// maxCells bounds the LCS table; beyond, the diff shows every line replaced.
const maxCells = 4_000_000
// Diff compares the current content of a file with the content to write.
func Diff(oldText, newText []byte) []Line {
a, b := lines(oldText), lines(newText)
ops := editScript(a, b)
// group the changes into hunks with their context
var out []Line
n := len(ops)
i := 0
for i < n {
if ops[i].Op == " " {
i++
continue
}
start := i - diffContext
if start < 0 {
start = 0
}
end := i
for end < n {
if ops[end].Op != " " {
end++
continue
}
// a run of context: the hunk ends when it is longer than 2*context
j := end
for j < n && ops[j].Op == " " {
j++
}
if j == n || j-end > 2*diffContext {
end += diffContext
if end > n {
end = n
}
break
}
end = j
}
oa, ob := ops[start].A, ops[start].B
ca, cb := 0, 0
for _, o := range ops[start:end] {
if o.Op != "+" {
ca++
}
if o.Op != "-" {
cb++
}
}
out = append(out, Line{Op: "@", Text: hunkHeader(oa, ca, ob, cb)})
for _, o := range ops[start:end] {
out = append(out, Line{Op: o.Op, Text: o.Text})
}
i = end
}
return out
}
type editOp struct {
Op string
Text string
A, B int // line numbers (1-based) where this op starts in a and b
}
func editScript(a, b []string) []editOp {
var ops []editOp
if len(a)*len(b) > maxCells {
for i, l := range a {
ops = append(ops, editOp{"-", l, i + 1, 1})
}
for i, l := range b {
ops = append(ops, editOp{"+", l, len(a) + 1, i + 1})
}
return ops
}
// lcs[i][j]: length of the LCS of a[i:] and b[j:]
lcs := make([][]int, len(a)+1)
for i := range lcs {
lcs[i] = make([]int, len(b)+1)
}
for i := len(a) - 1; i >= 0; i-- {
for j := len(b) - 1; j >= 0; j-- {
if a[i] == b[j] {
lcs[i][j] = lcs[i+1][j+1] + 1
} else if lcs[i+1][j] >= lcs[i][j+1] {
lcs[i][j] = lcs[i+1][j]
} else {
lcs[i][j] = lcs[i][j+1]
}
}
}
i, j := 0, 0
for i < len(a) || j < len(b) {
switch {
case i < len(a) && j < len(b) && a[i] == b[j]:
ops = append(ops, editOp{" ", a[i], i + 1, j + 1})
i++
j++
case j < len(b) && (i == len(a) || lcs[i][j+1] > lcs[i+1][j]):
ops = append(ops, editOp{"+", b[j], i + 1, j + 1})
j++
default:
ops = append(ops, editOp{"-", a[i], i + 1, j + 1})
i++
}
}
return ops
}
func lines(b []byte) []string {
s := string(b)
if s == "" {
return nil
}
s = strings.TrimSuffix(s, "\n")
return strings.Split(s, "\n")
}
func hunkHeader(a, ca, b, cb int) string {
r := func(start, n int) string {
if n == 0 {
start--
}
if n == 1 {
return strconv.Itoa(start)
}
return strconv.Itoa(start) + "," + strconv.Itoa(n)
}
return "@@ -" + r(a, ca) + " +" + r(b, cb) + " @@"
}
internal/nodesync/items.go
+100
View File
@@ -0,0 +1,100 @@
// Package nodesync replicates the system configuration of one PVE node (the
// master) to other nodes (the targets), of the same cluster or not. It is the
// second replication mode of pve-confsync, next to the cluster to cluster one
// (package engine). What the PVE API covers goes through the API: the network
// interfaces (/nodes/{node}/network, written in interfaces.new then reloaded)
// and the DNS servers (/nodes/{node}/dns). The other files go through SSH as
// root on the node: multipath, chrony, sysctl, interfaces.d. Same pipeline as
// the engine: read, normalize, diff, plan, apply with backup, reload, verify.
package nodesync
import "fmt"
// ItemKind tells how an item is read and written.
type ItemKind string
const (
KindNetwork ItemKind = "network" // API /nodes/{node}/network
KindDNS ItemKind = "dns" // API /nodes/{node}/dns
KindFile ItemKind = "file" // files over SSH
)
// Item is one replicable element.
type Item struct {
ID string `json:"id"`
Kind ItemKind `json:"kind"`
// Files: paths or glob patterns read on the nodes, and paths never touched.
Paths []string `json:"paths,omitempty"`
Exclude []string `json:"exclude,omitempty"`
// Glob items hold several files: a file only on the target is kept, or
// deleted with the prune option. Single file items are never deleted.
Glob bool `json:"glob,omitempty"`
// Additive files (multipath wwids) only get the lines of the master they
// miss: nothing is removed from the target.
Additive bool `json:"additive,omitempty"`
// Reload is the command run on the node after a write (empty: none).
Reload string `json:"reload,omitempty"`
// Network items reload the network of the node: typed confirmation,
// automatic restore when the node is unreachable afterwards.
Network bool `json:"network,omitempty"`
}
// Items is the catalog, in apply order (reloads follow this order, the
// network last).
var Items = []Item{
{ID: "sysctl", Kind: KindFile, Paths: []string{"/etc/sysctl.d/*.conf"}, Glob: true, Reload: "sysctl --system"},
{ID: "chrony", Kind: KindFile, Paths: []string{"/etc/chrony/chrony.conf", "/etc/chrony/sources.d/*.sources"}, Glob: true,
Reload: "systemctl restart chrony"},
{ID: "multipath", Kind: KindFile, Paths: []string{"/etc/multipath.conf"}, Reload: "multipathd reconfigure"},
{ID: "wwids", Kind: KindFile, Paths: []string{"/etc/multipath/wwids"}, Additive: true, Reload: "multipathd reconfigure"},
{ID: "dns", Kind: KindDNS},
// interfaces.d/sdn is generated by the SDN of PVE: never replicated
{ID: "interfaces-d", Kind: KindFile, Paths: []string{"/etc/network/interfaces.d/*"}, Exclude: []string{"/etc/network/interfaces.d/sdn"},
Glob: true, Reload: "ifreload -a", Network: true},
{ID: "network", Kind: KindNetwork, Network: true},
}
// ItemByID returns an item of the catalog.
func ItemByID(id string) (Item, error) {
for _, it := range Items {
if it.ID == id {
return it, nil
}
}
return Item{}, fmt.Errorf("unknown item %q", id)
}
func itemRank(id string) int {
for i, it := range Items {
if it.ID == id {
return i
}
}
return len(Items)
}
// Policy of a field of an API item (network, dns).
type Policy string
const (
Sync Policy = "sync" // copied from the master
Ignore Policy = "ignore" // never written
Override Policy = "override" // only written from the target mapping
)
// defaultPolicy: the addresses and gateways are those of each node; they are
// only written from the mapping of a target.
func defaultPolicy(item, field string) Policy {
if item == "network" {
switch field {
case "cidr", "cidr6", "gateway", "gateway6":
return Override
}
}
return Sync
}
// DefaultPolicies lists the fields whose default is not "sync", for the UI.
var DefaultPolicies = map[string]map[string]Policy{
"network": {"cidr": Override, "cidr6": Override, "gateway": Override, "gateway6": Override},
}
internal/nodesync/nodesync_test.go
+304
View File
@@ -0,0 +1,304 @@
package nodesync
import (
"context"
"crypto/sha256"
"encoding/hex"
"strings"
"testing"
"time"
"pve-confsync/internal/config"
"pve-confsync/internal/pve/pvetest"
"pve-confsync/internal/sshx"
)
func init() {
ProbeWait, TaskPoll = time.Millisecond, time.Millisecond
}
// endpoint connects to a node of a fake cluster like the web UI does.
func endpoint(t *testing.T, s *pvetest.Server, cluster string, n *pvetest.Node) Endpoint {
t.Helper()
sum := sha256.Sum256(s.Certificate().Raw)
c, err := config.Cluster{Name: cluster, URL: s.URL, User: "root@pam", Password: pvetest.Password,
Fingerprint: hex.EncodeToString(sum[:])}.PVE()
if err != nil {
t.Fatal(err)
}
host, port := n.SSHAddr()
o := sshx.Options{Host: host, Port: port, Password: pvetest.Password, HostKey: n.SSHHostKey()}
sh, err := sshx.New(o)
if err != nil {
t.Fatal(err)
}
return Endpoint{Key: cluster + "/" + n.Name, Cluster: cluster, Node: n.Name, API: c, Shell: sh}
}
const multipath = "defaults {\n user_friendly_names yes\n find_multipaths no\n}\nblacklist {\n devnode \"^sda\"\n}\n"
// lab: one cluster, a master pve1 and a target pve2 with differences.
func lab(t *testing.T) (*pvetest.Server, *pvetest.Node, *pvetest.Node, map[string]Endpoint) {
s := pvetest.New("9.1.4")
t.Cleanup(s.Close)
m, c := s.AddNode("pve1"), s.AddNode("pve2")
m.Net = map[string]map[string]any{
"eno1": {"type": "eth", "mtu": "9000"},
"eno2": {"type": "eth", "mtu": "9000"},
"bond0": {"type": "bond", "slaves": "eno1 eno2", "bond_mode": "802.3ad", "bond_xmit_hash_policy": "layer3+4", "mtu": "9000"},
"vmbr0": {"type": "bridge", "bridge_ports": "bond0", "bridge_vlan_aware": "1", "cidr": "10.0.0.11/24", "gateway": "10.0.0.1", "autostart": "1"},
"vmbr1": {"type": "bridge", "bridge_ports": "bond0.50", "cidr": "10.5.0.11/24", "comments": "stockage"},
"bond0.50": {"type": "vlan", "mtu": "9000"},
}
// the target has other NIC names and an active-backup bond
c.Net = map[string]map[string]any{
"enp1s0f0": {"type": "eth"},
"enp1s0f1": {"type": "eth"},
"bond0": {"type": "bond", "slaves": "enp1s0f0 enp1s0f1", "bond_mode": "active-backup"},
"vmbr0": {"type": "bridge", "bridge_ports": "bond0", "cidr": "10.0.0.12/24", "gateway": "10.0.0.1", "autostart": "1"},
}
m.DNS = map[string]string{"search": "lab.local", "dns1": "10.0.0.53", "dns2": "10.0.0.54"}
c.DNS = map[string]string{"search": "lab.local", "dns1": "8.8.8.8"}
m.Files["/etc/multipath.conf"] = pvetest.FakeFile{Content: multipath}
m.Files["/etc/multipath/wwids"] = pvetest.FakeFile{Content: "# wwids\n/3600a0980383030/\n/3600a0980383031/\n"}
c.Files["/etc/multipath/wwids"] = pvetest.FakeFile{Content: "# wwids\n/3600a0980383031/\n/36000local/\n"}
m.Files["/etc/sysctl.d/90-net.conf"] = pvetest.FakeFile{Content: "net.core.rmem_max = 16777216\n"}
c.Files["/etc/sysctl.d/90-net.conf"] = pvetest.FakeFile{Content: "net.core.rmem_max = 8388608\n", Mode: "600"}
c.Files["/etc/sysctl.d/99-local.conf"] = pvetest.FakeFile{Content: "vm.swappiness = 10\n"}
m.Files["/etc/network/interfaces.d/sdn"] = pvetest.FakeFile{Content: "auto vnet1\n"}
eps := map[string]Endpoint{"lab/pve1": endpoint(t, s, "lab", m), "lab/pve2": endpoint(t, s, "lab", c)}
return s, m, c, eps
}
func profile() *Profile {
return &Profile{Name: "stockage", Master: "lab/pve1", Targets: []string{"lab/pve2"},
Items: []string{"network", "dns", "multipath", "wwids", "sysctl", "interfaces-d"},
Mapping: map[string]TargetMap{"lab/pve2": {
NICs: map[string]string{"eno1": "enp1s0f0", "eno2": "enp1s0f1"},
Values: map[string]map[string]string{"network/vmbr1": {"cidr": "10.5.0.12/24"}},
}}}
}
func keys(p Plan) []string {
var out []string
for _, c := range p.Changes {
out = append(out, string(c.Op)+" "+c.Key())
}
return out
}
func TestPlan(t *testing.T) {
_, _, _, eps := lab(t)
prof := profile()
st := Analyze(context.Background(), prof, eps, nil)
if st.Master.Error != "" || st.Targets[0].Error != "" {
t.Fatal(st.Master.Error, st.Targets[0].Error)
}
p := st.Targets[0].Plan
if len(p.Errors) > 0 {
t.Fatalf("errors: %+v", p.Errors)
}
got := strings.Join(keys(p), "\n")
want := strings.Join([]string{
"update sysctl//etc/sysctl.d/90-net.conf",
"create multipath//etc/multipath.conf",
"update wwids//etc/multipath/wwids",
"update dns/dns",
"update network/enp1s0f0",
"update network/enp1s0f1",
"update network/bond0",
"create network/bond0.50",
"update network/vmbr0",
"create network/vmbr1",
}, "\n")
if got != want {
t.Fatalf("plan:\n%s\nwant:\n%s", got, want)
}
for _, c := range p.Changes {
switch c.Key() {
case "network/bond0":
if c.Set["slaves"] != "enp1s0f0 enp1s0f1" && c.Set["slaves"] != nil {
t.Errorf("slaves not renamed: %v", c.Set["slaves"])
}
if c.Set["bond_mode"] != "802.3ad" {
t.Errorf("bond0: %+v", c.Set)
}
case "network/vmbr0":
// the address of the target is kept (override without mapping)
if _, ok := c.Set["cidr"]; ok || c.Full["cidr"] != "10.0.0.12/24" {
t.Errorf("vmbr0: set %+v full %+v", c.Set, c.Full)
}
case "network/vmbr1":
if c.Full["cidr"] != "10.5.0.12/24" || c.Full["bridge_ports"] != "bond0.50" {
t.Errorf("vmbr1: %+v", c.Full)
}
case "wwids//etc/multipath/wwids":
// additive: the local WWID of the target stays
if string(c.Content) != "# wwids\n/3600a0980383031/\n/36000local/\n/3600a0980383030/\n" {
t.Errorf("wwids: %q", c.Content)
}
case "sysctl//etc/sysctl.d/90-net.conf":
if c.Mode != "600" || len(c.Diff) == 0 {
t.Errorf("sysctl: mode %s diff %+v", c.Mode, c.Diff)
}
}
}
// the file only on the target is kept without prune; interfaces.d/sdn is never read
for _, r := range st.Targets[0].Rows {
if r.ID == "/etc/sysctl.d/99-local.conf" && r.Status != StatusExtra {
t.Errorf("99-local.conf: %s", r.Status)
}
if strings.Contains(r.ID, "interfaces.d/sdn") {
t.Errorf("sdn file listed")
}
}
if !p.Network() {
t.Error("network plan not flagged")
}
if strings.Join(p.Reloads(), ",") != "sysctl --system,multipathd reconfigure,ifreload -a" {
t.Errorf("reloads: %v", p.Reloads())
}
}
func TestApplyAndVerify(t *testing.T) {
_, _, c, eps := lab(t)
prof := profile()
st := Analyze(context.Background(), prof, eps, nil)
tp := st.Plans(nil)[0]
var steps []string
if err := st.Apply(context.Background(), tp, nil, func(s Step) {
steps = append(steps, s.Action+" "+s.Error)
}); err != nil {
t.Fatalf("apply: %v\n%s", err, strings.Join(steps, "\n"))
}
if c.Net["vmbr0"]["cidr"] != "10.0.0.12/24" || c.Net["vmbr1"]["cidr"] != "10.5.0.12/24" {
t.Errorf("addresses: vmbr0 %v vmbr1 %v", c.Net["vmbr0"], c.Net["vmbr1"])
}
if c.Net["vmbr0"]["bridge_vlan_aware"] != "1" || c.Pending != nil {
t.Errorf("vmbr0 %v pending %v", c.Net["vmbr0"], c.Pending)
}
if c.DNS["dns2"] != "10.0.0.54" || c.Files["/etc/multipath.conf"].Content != multipath {
t.Errorf("dns %v multipath %q", c.DNS, c.Files["/etc/multipath.conf"].Content)
}
if len(c.Armed) != 0 {
t.Errorf("restore timer still armed: %v", c.Armed)
}
if got := strings.Join(c.Reloads, ","); got != "sysctl --system,multipathd reconfigure,network" {
t.Errorf("reloads: %s", got)
}
if len(c.Backups()) != 1 {
t.Errorf("backups: %v", c.Backups())
}
st = Analyze(context.Background(), prof, eps, nil)
if p := st.Targets[0].Plan; !p.Empty() {
t.Errorf("still to do: %v", keys(p))
}
}
func TestReloadFailureRestores(t *testing.T) {
_, _, c, eps := lab(t)
prof := profile()
prof.Items = []string{"multipath", "sysctl"}
c.FailOn = "reload multipathd reconfigure"
st := Analyze(context.Background(), prof, eps, nil)
err := st.Apply(context.Background(), st.Plans(nil)[0], nil, func(Step) {})
if err == nil || !strings.Contains(err.Error(), "multipathd reconfigure") {
t.Fatalf("err = %v", err)
}
if _, ok := c.Files["/etc/multipath.conf"]; ok {
t.Error("multipath.conf not removed by the restore")
}
if c.Files["/etc/sysctl.d/90-net.conf"].Content != "net.core.rmem_max = 8388608\n" {
t.Errorf("sysctl not restored: %q", c.Files["/etc/sysctl.d/90-net.conf"].Content)
}
}
func TestUnreachableAfterReloadKeepsTimer(t *testing.T) {
_, _, c, eps := lab(t)
prof := profile()
prof.Items = []string{"network"}
st := Analyze(context.Background(), prof, eps, nil)
c.FailOn = "probe"
err := st.Apply(context.Background(), st.Plans(nil)[0], nil, func(Step) {})
if err == nil || !strings.Contains(err.Error(), "unreachable") {
t.Fatalf("err = %v", err)
}
if len(c.Armed) != 1 {
t.Fatalf("timer: %v", c.Armed)
}
// systemd runs the restore: the previous interfaces come back
c.Fire()
if c.Net["bond0"]["bond_mode"] != "active-backup" || c.Net["vmbr1"] != nil {
t.Errorf("not restored: %v", c.Net)
}
}
func TestStaleFileRefused(t *testing.T) {
_, _, c, eps := lab(t)
prof := profile()
prof.Items = []string{"sysctl"}
st := Analyze(context.Background(), prof, eps, nil)
c.Files["/etc/sysctl.d/90-net.conf"] = pvetest.FakeFile{Content: "changed by hand\n"}
err := st.Apply(context.Background(), st.Plans(nil)[0], nil, func(Step) {})
if err == nil || !strings.Contains(err.Error(), "changed since the dry-run") {
t.Fatalf("err = %v", err)
}
if c.Files["/etc/sysctl.d/90-net.conf"].Content != "changed by hand\n" {
t.Error("file overwritten")
}
}
func TestPendingNetworkBlocks(t *testing.T) {
_, _, c, eps := lab(t)
c.Pending = map[string]map[string]any{"vmbr0": {"type": "bridge"}}
st := Analyze(context.Background(), profile(), eps, nil)
if len(st.Targets[0].Plan.Errors) == 0 {
t.Fatal("pending changes not blocking")
}
}
func TestMissingNICIsAnError(t *testing.T) {
_, _, _, eps := lab(t)
prof := profile()
prof.Items = []string{"network"}
prof.Mapping = nil
st := Analyze(context.Background(), prof, eps, nil)
p := st.Targets[0].Plan
if len(p.Errors) != 2 || p.Errors[0].Key != "network/eno1" {
t.Fatalf("errors: %+v", p.Errors)
}
// the selection of another object is not blocked
if f := Filter(p, []string{"network/vmbr0"}); len(f.Errors) != 0 || len(f.Changes) != 1 {
t.Errorf("filter: %+v", f)
}
}
func TestPruneAndProtect(t *testing.T) {
_, m, _, eps := lab(t)
m.Files["/etc/sysctl.d/90-net.conf"] = pvetest.FakeFile{Content: "# host pve1\nnet.core.rmem_max = 8388608\n"}
prof := profile()
prof.Items = []string{"sysctl"}
prof.Prune = true
prof.Protect = map[string][]string{"sysctl": {"^# host "}}
st := Analyze(context.Background(), prof, eps, nil)
got := strings.Join(keys(st.Targets[0].Plan), ",")
// the protected line is absent on the target: dropped, the rest is equal
if got != "delete sysctl//etc/sysctl.d/99-local.conf" {
t.Fatalf("plan: %s", got)
}
}
func TestDiff(t *testing.T) {
d := Diff([]byte("a\nb\nc\nd\ne\nf\ng\nh\ni\n"), []byte("a\nb\nc\nd\nE\nf\ng\nh\ni\nj\n"))
var b strings.Builder
for _, l := range d {
b.WriteString(l.Op + l.Text + "\n")
}
want := "@@@ -2,8 +2,9 @@\n b\n c\n d\n-e\n+E\n f\n g\n h\n i\n+j\n"
if b.String() != want {
t.Fatalf("diff:\n%s\nwant:\n%s", b.String(), want)
}
if d := Diff(nil, []byte("x\n")); len(d) != 2 || d[0].Text != "@@ -0,0 +1 @@" {
t.Errorf("creation diff: %+v", d)
}
}
internal/nodesync/plan.go
+661
View File
@@ -0,0 +1,661 @@
package nodesync
import (
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"regexp"
"sort"
"strings"
"pve-confsync/internal/model"
)
// Profile is one node replication: a master node, target nodes, the
// replicated items and their rules.
type Profile struct {
Name string `json:"name"`
Master string `json:"master"` // "cluster/node"
Targets []string `json:"targets"` // "cluster/node"
Items []string `json:"items"`
// Prune deletes on the targets what the master does not have: network
// interfaces (never the physical ones) and the files of the glob items.
Prune bool `json:"prune"`
// Policies of the fields of the API items (item -> field -> policy),
// over the defaults (addresses and gateways: override).
Policies map[string]map[string]Policy `json:"policies"`
// Protect: per file item, regular expressions of the lines kept from the
// target (hostname, local address…).
Protect map[string][]string `json:"protect"`
// Mapping: per target, its interface names and its own values.
Mapping map[string]TargetMap `json:"mapping"`
Position int `json:"position"`
}
// TargetMap holds what is specific to one target.
type TargetMap struct {
// NICs renames the interfaces of the master (eno1 -> enp1s0f0), in the
// interface names and in the fields that list interfaces.
NICs map[string]string `json:"nics,omitempty"`
// Values: "item/id" -> field -> value written on this target (cidr of
// vmbr0…), whatever the master has.
Values map[string]map[string]string `json:"values,omitempty"`
}
// Validate checks a profile.
func (p *Profile) Validate() error {
var errs []error
p.Name = strings.TrimSpace(p.Name)
if p.Name == "" {
errs = append(errs, errors.New("name is required"))
}
if p.Master == "" {
errs = append(errs, errors.New("master node is required"))
}
seen := map[string]bool{}
for _, t := range p.Targets {
if t == p.Master {
errs = append(errs, fmt.Errorf("target %s is the master", t))
}
if seen[t] {
errs = append(errs, fmt.Errorf("target %s listed twice", t))
}
seen[t] = true
}
if len(p.Items) == 0 {
errs = append(errs, errors.New("select at least one item"))
}
for _, id := range p.Items {
if _, err := ItemByID(id); err != nil {
errs = append(errs, err)
}
}
for item, fields := range p.Policies {
for f, pol := range fields {
switch pol {
case Sync, Ignore, Override:
default:
errs = append(errs, fmt.Errorf("%s.%s: unknown policy %q", item, f, pol))
}
if item == "network" && f == "type" && pol != Sync {
errs = append(errs, errors.New("network.type: always synchronized"))
}
}
}
for item, res := range p.Protect {
for _, re := range res {
if _, err := regexp.Compile(re); err != nil {
errs = append(errs, fmt.Errorf("%s: protected lines: %w", item, err))
}
}
}
return errors.Join(errs...)
}
// Policy resolves the policy of a field.
func (p *Profile) Policy(item, field string) Policy {
if item == "network" && field == "type" {
return Sync
}
if pol, ok := p.Policies[item][field]; ok {
return pol
}
return defaultPolicy(item, field)
}
// Has tells whether an item is replicated.
func (p *Profile) Has(item string) bool {
for _, x := range p.Items {
if x == item {
return true
}
}
return false
}
// Op is the action planned on one object.
type Op string
const (
OpCreate Op = "create"
OpUpdate Op = "update"
OpDelete Op = "delete"
)
// Change is one planned action: an interface or the DNS servers (Set, Unset,
// Old), or a file (Diff, Hash of the target file, content to write).
type Change struct {
Item string `json:"kind"` // item id ("kind" like the cluster plans, for the UI)
ID string `json:"id"` // interface, "dns" or file path
Op Op `json:"op"`
Set map[string]any `json:"set,omitempty"`
Unset []string `json:"unset,omitempty"`
Old map[string]any `json:"old,omitempty"`
Diff []Line `json:"diff,omitempty"`
Hash string `json:"hash,omitempty"` // current file on the target ("absent" when created)
Mode string `json:"mode,omitempty"`
Owner string `json:"owner,omitempty"`
// Full is the whole writable configuration of an interface after the
// change: PUT /nodes/{node}/network/{iface} recomputes the method and the
// families from the parameters, so every field is sent again.
Full map[string]any `json:"-"`
Content []byte `json:"-"`
}
// Key identifies the object of a change ("item/id").
func (c Change) Key() string { return c.Item + "/" + c.ID }
// Issue is a blocking error or a warning, about one object (Key) or the
// whole plan.
type Issue struct {
Key string `json:"key,omitempty"`
Message string `json:"message"`
}
// Plan is the list of changes for one target, in execution order.
type Plan struct {
Target string `json:"client"`
Changes []Change `json:"changes"`
Errors []Issue `json:"errors,omitempty"`
Warnings []Issue `json:"warnings,omitempty"`
}
// Empty tells whether there is nothing to do.
func (p Plan) Empty() bool { return len(p.Changes) == 0 }
// Network tells whether the plan reloads the network of the node.
func (p Plan) Network() bool {
for _, c := range p.Changes {
if it, _ := ItemByID(c.Item); it.Network {
return true
}
}
return false
}
// Reloads lists the commands run after the writes, in order.
func (p Plan) Reloads() []string {
seen := map[string]bool{}
var out []string
for _, it := range Items {
for _, c := range p.Changes {
if c.Item != it.ID {
continue
}
cmd := it.Reload
if it.Kind == KindNetwork {
cmd = "ifreload -a"
}
if cmd != "" && !seen[cmd] {
seen[cmd] = true
out = append(out, cmd)
}
break
}
}
return out
}
func (p *Plan) errorf(key, format string, a ...any) {
p.Errors = append(p.Errors, Issue{key, fmt.Sprintf(format, a...)})
}
func (p *Plan) warnf(key, format string, a ...any) {
p.Warnings = append(p.Warnings, Issue{key, fmt.Sprintf(format, a...)})
}
// MakePlan computes the changes that bring the target in line with the master.
func MakePlan(prof *Profile, master, target Snapshot) Plan {
plan := Plan{Target: target.Key}
tm := prof.Mapping[target.Key]
if target.Pending && prof.Has("network") {
plan.errorf("", "pending network changes on %s (interfaces.new): apply or revert them in PVE first", target.Key)
}
if master.Pending && prof.Has("network") {
plan.warnf("", "the master has pending network changes: its pending configuration is replicated")
}
for _, id := range prof.Items {
it, _ := ItemByID(id)
switch it.Kind {
case KindNetwork:
planNetwork(&plan, prof, tm, master, target)
case KindDNS:
planDNS(&plan, prof, tm, master, target)
case KindFile:
planFiles(&plan, prof, it, master.Files[it.ID], target.Files[it.ID])
}
}
sortChanges(plan.Changes)
return plan
}
// rank orders the interfaces: what is referenced first (ports, bonds, VLAN
// devices), bridges last; deletions run in the reverse order.
func netRank(typ string) int {
switch typ {
case "eth", "alias":
return 0
case "bond", "OVSBond":
return 1
case "vlan":
return 2
case "bridge", "OVSBridge":
return 3
}
return 4
}
func sortChanges(chs []Change) {
typ := func(c Change) string {
if t, ok := c.Full["type"].(string); ok {
return t
}
t, _ := c.Old["type"].(string)
return t
}
sort.SliceStable(chs, func(i, j int) bool {
a, b := chs[i], chs[j]
if (a.Op == OpDelete) != (b.Op == OpDelete) {
return b.Op == OpDelete
}
if a.Item != b.Item {
return itemRank(a.Item) < itemRank(b.Item)
}
if a.Item == "network" {
ra, rb := netRank(typ(a)), netRank(typ(b))
if a.Op == OpDelete {
ra, rb = rb, ra
}
if ra != rb {
return ra < rb
}
}
return a.ID < b.ID
})
}
// netLists are the fields naming other interfaces (space separated lists or
// one name): renamed with the NIC mapping of the target.
var netLists = map[string]bool{
"bridge_ports": true, "slaves": true, "ovs_ports": true, "ovs_bonds": true,
"bond-primary": true, "vlan-raw-device": true, "ovs_bridge": true,
}
// rename maps an interface name of the master to the target: eno1, and the
// VLAN interfaces built on it (eno1.100).
func rename(name string, nics map[string]string) string {
if to, ok := nics[name]; ok {
return to
}
if base, vlan, ok := strings.Cut(name, "."); ok {
if to, ok := nics[base]; ok {
return to + "." + vlan
}
}
return name
}
func renameList(v string, nics map[string]string) string {
parts := strings.Fields(v)
for i, p := range parts {
parts[i] = rename(p, nics)
}
return strings.Join(parts, " ")
}
func planNetwork(plan *Plan, prof *Profile, tm TargetMap, master, target Snapshot) {
cur := map[string]model.Object{}
for _, o := range target.Network {
cur[o.ID] = o
}
inMaster := map[string]bool{}
for _, m := range master.Network {
id := rename(m.ID, tm.NICs)
key := "network/" + id
inMaster[id] = true
desired := map[string]any{}
managed := map[string]bool{}
for f, v := range m.Fields {
if prof.Policy("network", f) != Sync {
continue
}
if netLists[f] {
v = renameList(v.(string), tm.NICs)
}
desired[f] = v
}
for f := range netWritable {
if prof.Policy("network", f) == Sync {
managed[f] = true
}
}
for f, v := range tm.Values[key] {
if prof.Policy("network", f) == Ignore || !netWritable[f] || f == "type" {
continue
}
desired[f] = v
managed[f] = true
}
typ, _ := desired["type"].(string)
o, exists := cur[id]
if !exists {
if physical(typ) {
plan.errorf(key, "physical interface %s absent on %s: add it to the NIC mapping of the target", id, target.Key)
continue
}
plan.Changes = append(plan.Changes, Change{Item: "network", ID: id, Op: OpCreate, Set: desired, Full: desired})
continue
}
if t, _ := o.Fields["type"].(string); t != typ {
plan.errorf(key, "type differs (%s on the target, %s on the master): re-create the interface by hand", t, typ)
continue
}
if mo, to := strings.Join(master.Options[m.ID], "\n"), strings.Join(target.Options[id], "\n"); mo != to {
plan.warnf(key, "options the API does not write differ (%s): not replicated", optionNames(master.Options[m.ID], target.Options[id]))
}
ch := Change{Item: "network", ID: id, Op: OpUpdate, Set: map[string]any{}, Old: map[string]any{}}
full := map[string]any{}
for f, v := range o.Fields {
full[f] = v
}
for f, v := range desired {
if !model.Equal(v, o.Fields[f]) {
ch.Set[f] = v
if old, ok := o.Fields[f]; ok {
ch.Old[f] = old
}
}
full[f] = v
}
for f := range managed {
if _, want := desired[f]; want {
continue
}
if old, has := o.Fields[f]; has {
ch.Unset = append(ch.Unset, f)
ch.Old[f] = old
delete(full, f)
}
}
sort.Strings(ch.Unset)
if len(ch.Set) > 0 || len(ch.Unset) > 0 {
ch.Full = full
plan.Changes = append(plan.Changes, ch)
}
}
if !prof.Prune {
return
}
for id, o := range cur {
if inMaster[id] {
continue
}
if t, _ := o.Fields["type"].(string); physical(t) {
continue
}
plan.Changes = append(plan.Changes, Change{Item: "network", ID: id, Op: OpDelete, Old: o.Fields})
}
}
// optionNames gives the option keywords that differ (bond-miimon, post-up…).
func optionNames(a, b []string) string {
set := map[string]bool{}
count := map[string]int{}
for _, l := range a {
count[l]++
}
for _, l := range b {
count[l]--
}
for l, n := range count {
if n != 0 {
set[strings.Fields(l + " ")[0]] = true
}
}
var out []string
for k := range set {
out = append(out, k)
}
sort.Strings(out)
return strings.Join(out, ", ")
}
func planDNS(plan *Plan, prof *Profile, tm TargetMap, master, target Snapshot) {
const key = "dns/dns"
desired := map[string]any{}
for _, f := range dnsFields {
pol := prof.Policy("dns", f)
if v, ok := master.DNS[f]; ok && pol == Sync {
desired[f] = v
}
if pol != Sync {
if v, ok := target.DNS[f]; ok {
desired[f] = v // PUT /nodes/{node}/dns rewrites every server
}
}
if v, ok := tm.Values[key][f]; ok && pol != Ignore {
desired[f] = v
}
}
ch := Change{Item: "dns", ID: "dns", Op: OpUpdate, Set: map[string]any{}, Old: map[string]any{}, Full: desired}
for _, f := range dnsFields {
v, want := desired[f]
old, has := target.DNS[f]
switch {
case want && !model.Equal(v, old):
ch.Set[f] = v
if has {
ch.Old[f] = old
}
case !want && has:
ch.Unset = append(ch.Unset, f)
ch.Old[f] = old
}
}
if len(ch.Set) == 0 && len(ch.Unset) == 0 {
return
}
if desired["search"] == nil {
plan.errorf(key, "a search domain is required by the API (PUT /nodes/{node}/dns)")
return
}
plan.Changes = append(plan.Changes, ch)
}
func planFiles(plan *Plan, prof *Profile, it Item, master, target map[string]File) {
var protect []*regexp.Regexp
for _, s := range prof.Protect[it.ID] {
if re, err := regexp.Compile(s); err == nil {
protect = append(protect, re)
}
}
if len(master) == 0 && !it.Glob {
plan.warnf(it.ID+"/"+it.Paths[0], "%s absent on the master: not replicated", it.Paths[0])
return
}
paths := make([]string, 0, len(master))
for p := range master {
paths = append(paths, p)
}
sort.Strings(paths)
for _, p := range paths {
m := master[p]
t, exists := target[p]
var content []byte
switch {
case it.Additive && exists:
content = additive(t.Content, m.Content)
case exists:
content = protectLines(m.Content, t.Content, protect)
default:
content = protectLines(m.Content, nil, protect)
}
if exists && hash(content) == t.Hash {
continue
}
ch := Change{Item: it.ID, ID: p, Content: content, Mode: m.Mode, Owner: m.Owner, Hash: "absent", Op: OpCreate}
var old []byte
if exists {
// an update keeps the mode and owner of the target file
ch.Op, ch.Hash, ch.Mode, ch.Owner, old = OpUpdate, t.Hash, t.Mode, t.Owner, t.Content
}
ch.Diff = Diff(old, content)
plan.Changes = append(plan.Changes, ch)
}
if !prof.Prune || !it.Glob {
return
}
for p, t := range target {
if _, ok := master[p]; ok {
continue
}
plan.Changes = append(plan.Changes, Change{Item: it.ID, ID: p, Op: OpDelete, Hash: t.Hash, Diff: Diff(t.Content, nil)})
}
}
// protectLines: each line of the master matching a protected expression takes
// the value of the first line of the target matching the same expression
// (dropped when the target has none).
func protectLines(master, target []byte, protect []*regexp.Regexp) []byte {
if len(protect) == 0 {
return master
}
tl := lines(target)
used := map[int]bool{}
var out []string
for _, l := range lines(master) {
var re *regexp.Regexp
for _, r := range protect {
if r.MatchString(l) {
re = r
break
}
}
if re == nil {
out = append(out, l)
continue
}
for i, x := range tl {
if !used[i] && re.MatchString(x) {
used[i] = true
out = append(out, x)
break
}
}
}
return joinLines(out)
}
// additive appends to the target the lines of the master it does not have.
func additive(target, master []byte) []byte {
tl := lines(target)
have := map[string]bool{}
for _, l := range tl {
have[l] = true
}
out := tl
for _, l := range lines(master) {
if !have[l] {
have[l] = true
out = append(out, l)
}
}
return joinLines(out)
}
func joinLines(l []string) []byte {
if len(l) == 0 {
return nil
}
return []byte(strings.Join(l, "\n") + "\n")
}
// Filter keeps the changes of the given objects ("item/id"), plus the
// interfaces they need that the plan creates (ports of a bridge, raw device
// of a VLAN). Issues about other objects do not block the selection.
func Filter(p Plan, keys []string) Plan {
if len(keys) == 0 {
return p
}
byKey := map[string]Change{}
for _, c := range p.Changes {
byKey[c.Key()] = c
}
want := map[string]bool{}
for _, k := range keys {
want[k] = true
}
for changed := true; changed; {
changed = false
for k := range want {
c, ok := byKey[k]
if !ok || c.Item != "network" || c.Op == OpDelete {
continue
}
for f, v := range c.Full {
if !netLists[f] {
continue
}
for _, n := range strings.Fields(v.(string)) {
dep := "network/" + n
if d, ok := byKey[dep]; ok && d.Op == OpCreate && !want[dep] {
want[dep] = true
changed = true
}
}
}
}
}
out := Plan{Target: p.Target}
for _, e := range p.Errors {
if e.Key == "" || want[e.Key] {
out.Errors = append(out.Errors, e)
}
}
for _, w := range p.Warnings {
if w.Key == "" || want[w.Key] {
out.Warnings = append(out.Warnings, w)
}
}
for _, c := range p.Changes {
if want[c.Key()] {
out.Changes = append(out.Changes, c)
}
}
return out
}
// Digest identifies the content of a plan: the apply only runs when the plan
// computed again is the one reviewed in the dry-run.
func (p Plan) Digest() string {
type ch struct {
Op Op `json:"op"`
Key string `json:"key"`
Set map[string]any `json:"set,omitempty"`
Unset []string `json:"unset,omitempty"`
Hash string `json:"hash,omitempty"`
New string `json:"new,omitempty"`
}
l := make([]ch, 0, len(p.Changes))
for _, c := range p.Changes {
x := ch{Op: c.Op, Key: c.Key(), Set: c.Set, Unset: c.Unset, Hash: c.Hash}
if c.Content != nil || c.Op != OpDelete && c.Hash != "" {
x.New = hash(c.Content)
}
l = append(l, x)
}
errs := make([]string, 0, len(p.Errors))
for _, e := range p.Errors {
errs = append(errs, e.Key+" "+e.Message)
}
sort.Strings(errs)
b, _ := json.Marshal(struct {
Target string `json:"target"`
Changes []ch `json:"changes"`
Errors []string `json:"errors"`
}{p.Target, l, errs})
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:12])
}
internal/nodesync/read.go
+337
View File
@@ -0,0 +1,337 @@
package nodesync
import (
"bufio"
"bytes"
"context"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"fmt"
"math/bits"
"net/url"
"path"
"sort"
"strconv"
"strings"
"time"
"pve-confsync/internal/model"
"pve-confsync/internal/pve"
)
// Shell runs a command as root on the node, stdin fed with in (sshx.Client).
type Shell interface {
Run(ctx context.Context, cmd string, in []byte) ([]byte, error)
}
// Endpoint is one node: the API of its cluster (calls proxied to the node)
// and SSH as root on the node itself.
type Endpoint struct {
Key string // "cluster/node"
Cluster string
Node string
API *pve.Client
Shell Shell // nil: SSH host key not recorded yet
// Dial opens a new SSH access, to check that the node still answers after
// a network reload (nil: Shell is used).
Dial func() (Shell, error)
}
// File is a file read on a node.
type File struct {
Path string `json:"path"`
Mode string `json:"mode"` // octal, e.g. 644
Owner string `json:"owner"` // user:group
Hash string `json:"hash"` // SHA-256 of the content
Content []byte `json:"-"`
}
// Snapshot is what was read on one node.
type Snapshot struct {
Key string `json:"key"`
Version string `json:"version,omitempty"`
Network []model.Object `json:"-"` // interfaces (id = iface), canonical fields
Options map[string][]string `json:"-"` // iface -> lines the API does not write (bond-miimon…)
DNS map[string]any `json:"-"`
Files map[string]map[string]File `json:"-"` // item -> path -> file
// Pending: /etc/network/interfaces.new exists (network changes not applied yet).
Pending bool `json:"pending,omitempty"`
Warnings []string `json:"warnings,omitempty"`
ReadAt time.Time `json:"readAt"`
}
// maxFile: larger files are not replicated (configuration files are small).
const maxFile = 1 << 20
// Read loads the given items from a node.
func Read(ctx context.Context, ep Endpoint, items []string) (Snapshot, error) {
snap := Snapshot{Key: ep.Key, Files: map[string]map[string]File{}, ReadAt: time.Now().UTC()}
v, err := NodeVersion(ctx, ep)
if err != nil {
return snap, err
}
snap.Version = v
var files []Item
for _, id := range items {
it, err := ItemByID(id)
if err != nil {
return snap, err
}
switch it.Kind {
case KindNetwork:
if snap.Network, snap.Options, err = readNetwork(ctx, ep); err != nil {
return snap, fmt.Errorf("%s: read network: %w", ep.Key, err)
}
case KindDNS:
if snap.DNS, err = readDNS(ctx, ep); err != nil {
return snap, fmt.Errorf("%s: read dns: %w", ep.Key, err)
}
case KindFile:
files = append(files, it)
}
}
// SSH is needed for the files, and for the pending network changes
if ep.Shell == nil {
return snap, fmt.Errorf("%s: SSH host key not recorded yet, test the node first", ep.Key)
}
out, err := ep.Shell.Run(ctx, readScript(files), nil)
if err != nil {
return snap, fmt.Errorf("%s: read files: %w", ep.Key, err)
}
got, pending, warns, err := parseRead(out)
if err != nil {
return snap, fmt.Errorf("%s: read files: %w", ep.Key, err)
}
snap.Pending = pending
snap.Warnings = append(snap.Warnings, warns...)
for _, it := range files {
m := map[string]File{}
for p, f := range got {
if matchItem(it, p) {
m[p] = f
}
}
snap.Files[it.ID] = m
}
return snap, nil
}
// NodeVersion reads the pve-manager version of the node and refuses anything
// but PVE 9.x.
func NodeVersion(ctx context.Context, ep Endpoint) (string, error) {
var v struct {
Version string `json:"version"`
Release string `json:"release"`
}
if err := ep.API.Get(ctx, "nodes/"+url.PathEscape(ep.Node)+"/version", nil, &v); err != nil {
return "", fmt.Errorf("%s: version check: %w", ep.Key, err)
}
ref := v.Release
if ref == "" {
ref = v.Version
}
if major, err := strconv.Atoi(strings.SplitN(ref, ".", 2)[0]); err != nil || major != pve.SupportedMajor {
return v.Version, fmt.Errorf("%s: node runs PVE %q, only %d.x is supported", ep.Key, ref, pve.SupportedMajor)
}
return v.Version, nil
}
func matchItem(it Item, p string) bool {
for _, x := range it.Exclude {
if x == p {
return false
}
}
for _, pat := range it.Paths {
if ok, _ := path.Match(pat, p); ok {
return true
}
}
return false
}
// readScript lists the files of the items: one header line per file
// ("F <mode> <user:group> <path>", "L <size> <path>" when too large) then
// its content in base64 on one line; "PENDING" when interfaces.new exists.
// The first line is a marker the fake node of the tests understands.
func readScript(files []Item) string {
var pats []string
for _, it := range files {
pats = append(pats, it.Paths...)
}
var b strings.Builder
b.WriteString("# confsync:read\n")
if len(pats) > 0 {
fmt.Fprintf(&b, "for f in %s; do\n", strings.Join(pats, " "))
b.WriteString(` [ -f "$f" ] && [ ! -L "$f" ] || continue
s=$(stat -c %s "$f")
if [ "$s" -gt ` + strconv.Itoa(maxFile) + ` ]; then printf 'L %s %s\n' "$s" "$f"; continue; fi
printf 'F %s %s\n' "$(stat -c '%a %U:%G' "$f")" "$f"
base64 -w0 < "$f"; echo
done
`)
}
b.WriteString("[ -e /etc/network/interfaces.new ] && echo PENDING\ntrue\n")
return b.String()
}
func parseRead(out []byte) (map[string]File, bool, []string, error) {
files := map[string]File{}
pending := false
var warns []string
sc := bufio.NewScanner(bytes.NewReader(out))
sc.Buffer(make([]byte, 64<<10), 4*maxFile)
for sc.Scan() {
line := sc.Text()
switch {
case line == "PENDING":
pending = true
case strings.HasPrefix(line, "L "):
p := strings.SplitN(line, " ", 3)
if len(p) == 3 {
warns = append(warns, fmt.Sprintf("%s: %s bytes, too large, not replicated", p[2], p[1]))
}
case strings.HasPrefix(line, "F "):
p := strings.SplitN(line, " ", 4)
if len(p) != 4 || !sc.Scan() {
return nil, false, nil, fmt.Errorf("unexpected output %q", line)
}
data, err := base64.StdEncoding.DecodeString(strings.TrimSpace(sc.Text()))
if err != nil {
return nil, false, nil, fmt.Errorf("%s: %w", p[3], err)
}
if !safePath(p[3]) {
warns = append(warns, fmt.Sprintf("%s: unexpected characters in the path, not replicated", p[3]))
continue
}
files[p[3]] = File{Path: p[3], Mode: p[1], Owner: p[2], Hash: hash(data), Content: data}
}
}
return files, pending, warns, sc.Err()
}
// safePath: the paths go into shell scripts between single quotes.
func safePath(p string) bool {
if !strings.HasPrefix(p, "/") || strings.Contains(p, "..") {
return false
}
for _, r := range p {
if !(r == '/' || r == '.' || r == '-' || r == '_' || r == '+' || r == '@' ||
(r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9')) {
return false
}
}
return true
}
func hash(b []byte) string {
s := sha256.Sum256(b)
return hex.EncodeToString(s[:])
}
// ---- network (GET /nodes/{node}/network) ----
// netWritable are the parameters of POST/PUT /nodes/{node}/network[/{iface}]
// (pve-manager PVE/API2/Network.pm, $confdesc), in their cidr form: the GET
// returns address/netmask and cidr, the canonical form keeps cidr only.
var netWritable = map[string]bool{
"type": true, "comments": true, "comments6": true, "autostart": true,
"bridge_vlan_aware": true, "bridge_vids": true, "bridge_ports": true,
"ovs_ports": true, "ovs_tag": true, "ovs_options": true, "ovs_bridge": true, "ovs_bonds": true,
"slaves": true, "bond_mode": true, "bond-primary": true, "bond_xmit_hash_policy": true,
"vlan-raw-device": true, "vlan-id": true,
"gateway": true, "cidr": true, "gateway6": true, "cidr6": true, "mtu": true,
}
// netTypes are the interface types handled (fabric, vnet and unknown are not).
var netTypes = map[string]bool{
"eth": true, "bond": true, "bridge": true, "vlan": true, "alias": true,
"OVSBridge": true, "OVSBond": true, "OVSPort": true, "OVSIntPort": true,
}
// physical interfaces cannot be created: they must exist on the target.
func physical(typ string) bool { return typ == "eth" }
func readNetwork(ctx context.Context, ep Endpoint) ([]model.Object, map[string][]string, error) {
var raw []map[string]any
if err := ep.API.Get(ctx, "nodes/"+url.PathEscape(ep.Node)+"/network", nil, &raw); err != nil {
return nil, nil, err
}
var objs []model.Object
opts := map[string][]string{}
for _, r := range raw {
id := pve.Str(r["iface"])
typ := pve.Str(r["type"])
if id == "" || id == "lo" || !netTypes[typ] {
continue
}
f := map[string]any{}
for k, v := range r {
if !netWritable[k] || v == nil {
continue
}
s := strings.TrimRight(pve.Str(v), "\n")
if s == "" {
continue
}
f[k] = s
}
// a static address without cidr (older parsers): rebuild it
if _, ok := f["cidr"]; !ok {
if a, m := pve.Str(r["address"]), pve.Str(r["netmask"]); a != "" && m != "" {
f["cidr"] = a + "/" + maskBits(m)
}
}
if l := listOf(r["options"]); len(l) > 0 {
opts[id] = l
}
if l := listOf(r["options6"]); len(l) > 0 {
opts[id] = append(opts[id], l...)
}
objs = append(objs, model.Object{Kind: "network", ID: id, Fields: f})
}
sort.Slice(objs, func(i, j int) bool { return objs[i].ID < objs[j].ID })
return objs, opts, nil
}
func listOf(v any) []string {
l, _ := v.([]any)
out := make([]string, 0, len(l))
for _, e := range l {
out = append(out, pve.Str(e))
}
sort.Strings(out)
return out
}
// maskBits converts 255.255.255.0 to 24 (a prefix length is kept).
func maskBits(m string) string {
if !strings.Contains(m, ".") {
return m
}
n := 0
for _, p := range strings.Split(m, ".") {
b, _ := strconv.Atoi(p)
n += bits.OnesCount8(uint8(b))
}
return strconv.Itoa(n)
}
// ---- dns (GET /nodes/{node}/dns) ----
var dnsFields = []string{"search", "dns1", "dns2", "dns3"}
func readDNS(ctx context.Context, ep Endpoint) (map[string]any, error) {
var raw map[string]any
if err := ep.API.Get(ctx, "nodes/"+url.PathEscape(ep.Node)+"/dns", nil, &raw); err != nil {
return nil, err
}
out := map[string]any{}
for _, k := range dnsFields {
if s := pve.Str(raw[k]); s != "" {
out[k] = s
}
}
return out, nil
}
internal/nodesync/script_test.go
+77
View File
@@ -0,0 +1,77 @@
package nodesync
import (
"os"
"os/exec"
"os/user"
"path/filepath"
"strings"
"testing"
)
// TestPrepareScriptForReal runs the prepare script and restore.sh with the
// local shell in a temporary folder (every path prefixed with it).
func TestPrepareScriptForReal(t *testing.T) {
if _, err := exec.LookPath("sha256sum"); err != nil {
t.Skip("sha256sum not installed")
}
root := t.TempDir()
u, _ := user.Current()
g, _ := user.LookupGroupId(u.Gid)
owner := u.Username + ":" + g.Name
write := func(p, s string) {
if err := os.MkdirAll(filepath.Dir(root+p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(root+p, []byte(s), 0o644); err != nil {
t.Fatal(err)
}
}
read := func(p string) string {
b, err := os.ReadFile(root + p)
if err != nil {
return "<absent>"
}
return string(b)
}
write("/etc/sysctl.d/90-net.conf", "old\n")
write("/etc/sysctl.d/99-local.conf", "local\n")
write("/etc/network/interfaces", "auto lo\n")
plan := Plan{Changes: []Change{
{Item: "sysctl", ID: "/etc/sysctl.d/90-net.conf", Op: OpUpdate, Hash: hash([]byte("old\n")), Mode: "640", Owner: owner, Content: []byte("new\n")},
{Item: "multipath", ID: "/etc/multipath.conf", Op: OpCreate, Hash: "absent", Mode: "644", Owner: owner, Content: []byte("defaults {}\n")},
{Item: "sysctl", ID: "/etc/sysctl.d/99-local.conf", Op: OpDelete, Hash: hash([]byte("local\n"))},
{Item: "network", ID: "vmbr0", Op: OpUpdate},
}}
run := func(script string, in []byte) (string, error) {
cmd := exec.Command("sh", "-c", script)
cmd.Stdin = strings.NewReader(string(in))
out, err := cmd.CombinedOutput()
return string(out), err
}
const dir = "/var/lib/pve-confsync/backup/test"
if out, err := run(prepareScriptIn(dir, root), prepareInput(plan, true, false)); err != nil || strings.TrimSpace(out) != "ok" {
t.Fatalf("prepare: %v %s", err, out)
}
if read("/etc/sysctl.d/90-net.conf") != "new\n" || read("/etc/multipath.conf") != "defaults {}\n" || read("/etc/sysctl.d/99-local.conf") != "<absent>" {
t.Fatalf("writes: %q %q %q", read("/etc/sysctl.d/90-net.conf"), read("/etc/multipath.conf"), read("/etc/sysctl.d/99-local.conf"))
}
if fi, _ := os.Stat(root + "/etc/sysctl.d/90-net.conf"); fi.Mode().Perm() != 0o640 {
t.Errorf("mode %v", fi.Mode())
}
if m, _ := filepath.Glob(root + "/etc/sysctl.d/*.confsync.*"); len(m) > 0 {
t.Errorf("temporary files left: %v", m)
}
// a second run with the same plan: the files changed, nothing is written
if out, err := run(prepareScriptIn(dir+"2", root), prepareInput(plan, false, false)); err == nil || !strings.Contains(out, "changed since the dry-run") {
t.Errorf("stale check: %v %s", err, out)
}
write("/etc/network/interfaces.new", "pending\n")
if out, err := run("sh '"+root+dir+"/restore.sh'", nil); err != nil {
t.Fatalf("restore: %v %s", err, out)
}
if read("/etc/sysctl.d/90-net.conf") != "old\n" || read("/etc/multipath.conf") != "<absent>" || read("/etc/sysctl.d/99-local.conf") != "local\n" ||
read("/etc/network/interfaces") != "auto lo\n" || read("/etc/network/interfaces.new") != "<absent>" {
t.Errorf("restore: %q %q %q %q", read("/etc/sysctl.d/90-net.conf"), read("/etc/multipath.conf"), read("/etc/sysctl.d/99-local.conf"), read("/etc/network/interfaces.new"))
}
}
internal/pve/client.go
+182 -45
View File
@@ -1,6 +1,8 @@
// Package pve is a minimal Proxmox VE API client: API token auth, generic JSON,
// form-encoded writes, task polling and version check. No typed SDK on purpose:
// objects stay generic (map[string]any) and are interpreted by the adapters.
// Package pve is a minimal Proxmox VE API client: ticket auth with the root@pam
// credentials, generic JSON, form-encoded writes, task polling and version check.
// No typed SDK on purpose: objects stay generic (map[string]any) and are
// interpreted by the adapters. What the API does not cover (the password hashes
// of the pve realm) goes through a Shell, an SSH session on a node.
package pve
import (
@@ -15,10 +17,10 @@ import (
"io"
"net/http"
"net/url"
"os"
"sort"
"strconv"
"strings"
"sync"
"time"
)
@@ -26,23 +28,42 @@ import (
type Options struct {
Name string // cluster name in the config (for messages only)
BaseURL string // https://host:8006
TokenID string // user@realm!tokenname
TokenSecret string // never logged
CAFile string // PEM bundle to trust (optional)
CAPEM []byte // PEM bundle given inline (web UI); CAFile wins when both are set
User string // user@realm, root@pam by default
Password string // never logged
CAPEM []byte // PEM bundle to trust (optional)
Fingerprint string // SHA-256 of the leaf certificate, "AA:BB:..." (optional, pinning)
Insecure bool // skip TLS verification (lab only)
Timeout time.Duration
Shell Shell // SSH access to a node (optional)
}
// Shell runs a command on a node of the cluster (SSH), stdin fed with in.
type Shell interface {
Run(ctx context.Context, cmd string, in []byte) ([]byte, error)
}
// ticketTTL: PVE tickets live 2 hours (pve-access-control AccessControl.pm,
// $ticket_lifetime); a new one is requested well before.
const ticketTTL = 90 * time.Minute
// Client talks to the PVE API of one cluster.
type Client struct {
name string
base *url.URL
auth string
user string
password string
http *http.Client
shell Shell
mu sync.Mutex
ticket string
csrf string
ticketAt time.Time
}
// ErrNoShell: the operation needs an SSH access to a node.
var ErrNoShell = errors.New("SSH access required")
// APIError is a non-2xx answer of the API.
type APIError struct {
Method, Path string
@@ -75,41 +96,15 @@ func New(o Options) (*Client, error) {
if err != nil || u.Scheme == "" || u.Host == "" {
return nil, fmt.Errorf("cluster %s: invalid url %q", o.Name, o.BaseURL)
}
if o.TokenID == "" || o.TokenSecret == "" {
return nil, fmt.Errorf("cluster %s: token id and secret are required", o.Name)
if o.User == "" {
o.User = "root@pam"
}
tlsCfg := &tls.Config{MinVersion: tls.VersionTLS12}
switch {
case o.Fingerprint != "":
want := normalizeFingerprint(o.Fingerprint)
// Self-signed PVE certificates: pin the leaf certificate instead of a CA.
tlsCfg.InsecureSkipVerify = true
tlsCfg.VerifyPeerCertificate = func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return errors.New("no peer certificate")
if o.Password == "" {
return nil, fmt.Errorf("cluster %s: the password of %s is required", o.Name, o.User)
}
sum := sha256.Sum256(raw[0])
if got := hex.EncodeToString(sum[:]); got != want {
return fmt.Errorf("certificate fingerprint mismatch (got %s)", got)
}
return nil
}
case o.CAFile != "" || len(o.CAPEM) > 0:
pem, src := o.CAPEM, "the CA certificate"
if o.CAFile != "" {
var err error
if pem, err = os.ReadFile(o.CAFile); err != nil {
return nil, fmt.Errorf("cluster %s: %w", o.Name, err)
}
src = o.CAFile
}
pool := x509.NewCertPool()
if !pool.AppendCertsFromPEM(pem) {
return nil, fmt.Errorf("cluster %s: no certificate in %s", o.Name, src)
}
tlsCfg.RootCAs = pool
case o.Insecure:
tlsCfg.InsecureSkipVerify = true
tlsCfg, err := TLSConfig(o.Name, o.CAPEM, o.Fingerprint, o.Insecure)
if err != nil {
return nil, err
}
timeout := o.Timeout
if timeout == 0 {
@@ -118,7 +113,9 @@ func New(o Options) (*Client, error) {
return &Client{
name: o.Name,
base: u,
auth: "PVEAPIToken=" + o.TokenID + "=" + o.TokenSecret,
user: o.User,
password: o.Password,
shell: o.Shell,
http: &http.Client{
Timeout: timeout,
Transport: &http.Transport{TLSClientConfig: tlsCfg, Proxy: http.ProxyFromEnvironment},
@@ -126,13 +123,127 @@ func New(o Options) (*Client, error) {
}, nil
}
// TLSConfig builds the TLS settings of a cluster: certificate pinning (self-signed
// PVE certificates), a CA bundle, the system authorities, or no verification.
func TLSConfig(name string, caPEM []byte, fingerprint string, insecure bool) (*tls.Config, error) {
tlsCfg := &tls.Config{MinVersion: tls.VersionTLS12}
switch {
case fingerprint != "":
want := normalizeFingerprint(fingerprint)
// Self-signed PVE certificates: pin the leaf certificate instead of a CA.
tlsCfg.InsecureSkipVerify = true
tlsCfg.VerifyPeerCertificate = func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return errors.New("no peer certificate")
}
sum := sha256.Sum256(raw[0])
if got := hex.EncodeToString(sum[:]); got != want {
return fmt.Errorf("certificate fingerprint mismatch (got %s)", FormatFingerprint(got))
}
return nil
}
case len(caPEM) > 0:
pool := x509.NewCertPool()
if !pool.AppendCertsFromPEM(caPEM) {
return nil, fmt.Errorf("cluster %s: no certificate in the CA bundle", name)
}
tlsCfg.RootCAs = pool
case insecure:
tlsCfg.InsecureSkipVerify = true
}
return tlsCfg, nil
}
// PeerFingerprint connects to the API and returns the SHA-256 fingerprint of the
// certificate it presents, without verifying it: the administrator pins it on
// the first connection (trust on first use) and sees it in the cluster list.
func PeerFingerprint(ctx context.Context, baseURL string) (string, error) {
u, err := url.Parse(baseURL)
if err != nil || u.Host == "" {
return "", fmt.Errorf("invalid url %q", baseURL)
}
host := u.Host
if u.Port() == "" {
host += ":8006"
}
d := tls.Dialer{Config: &tls.Config{MinVersion: tls.VersionTLS12, InsecureSkipVerify: true}}
conn, err := d.DialContext(ctx, "tcp", host)
if err != nil {
return "", err
}
defer conn.Close()
certs := conn.(*tls.Conn).ConnectionState().PeerCertificates
if len(certs) == 0 {
return "", errors.New("no peer certificate")
}
sum := sha256.Sum256(certs[0].Raw)
return FormatFingerprint(hex.EncodeToString(sum[:])), nil
}
// FormatFingerprint prints a SHA-256 fingerprint like PVE does ("AA:BB:...").
func FormatFingerprint(fp string) string {
fp = strings.ToUpper(normalizeFingerprint(fp))
var b strings.Builder
for i := 0; i < len(fp); i += 2 {
if i > 0 {
b.WriteByte(':')
}
b.WriteString(fp[i:min(i+2, len(fp))])
}
return b.String()
}
// Name returns the cluster name given in the options.
func (c *Client) Name() string { return c.name }
// Shell returns the SSH access to a node, or ErrNoShell.
func (c *Client) Shell() (Shell, error) {
if c.shell == nil {
return nil, fmt.Errorf("cluster %s: %w", c.name, ErrNoShell)
}
return c.shell, nil
}
func normalizeFingerprint(s string) string {
return strings.ToLower(strings.NewReplacer(":", "", " ", "").Replace(s))
}
// login gets a ticket and a CSRF token (POST /access/ticket). Accounts with a
// second factor are refused: the tool cannot answer the TFA challenge.
func (c *Client) login(ctx context.Context) error {
var res struct {
Ticket string `json:"ticket"`
CSRF string `json:"CSRFPreventionToken"`
NeedTFA any `json:"NeedTFA"`
}
f := url.Values{"username": {c.user}, "password": {c.password}}
if err := c.send(ctx, http.MethodPost, "access/ticket", nil, f, &res, false); err != nil {
var ae *APIError
if errors.As(err, &ae) && ae.Status == http.StatusUnauthorized {
return fmt.Errorf("cluster %s: authentication of %s failed", c.name, c.user)
}
return err
}
if Str(res.NeedTFA) == "1" {
return fmt.Errorf("cluster %s: %s has a second factor (TFA), not supported", c.name, c.user)
}
if res.Ticket == "" {
return fmt.Errorf("cluster %s: no ticket in the login answer", c.name)
}
c.ticket, c.csrf, c.ticketAt = res.Ticket, res.CSRF, time.Now()
return nil
}
// Login checks the credentials (and renews the ticket when needed).
func (c *Client) Login(ctx context.Context) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.ticket != "" && time.Since(c.ticketAt) < ticketTTL {
return nil
}
return c.login(ctx)
}
// Get reads path (relative to /api2/json) and decodes "data" into out.
func (c *Client) Get(ctx context.Context, path string, query url.Values, out any) error {
return c.do(ctx, http.MethodGet, path, query, nil, out)
@@ -153,6 +264,25 @@ func (c *Client) Delete(ctx context.Context, path string, params url.Values, out
}
func (c *Client) do(ctx context.Context, method, path string, query, form url.Values, out any) error {
if err := c.Login(ctx); err != nil {
return err
}
err := c.send(ctx, method, path, query, form, out, true)
var ae *APIError
if errors.As(err, &ae) && ae.Status == http.StatusUnauthorized {
// ticket expired early (key rotation on the cluster): log in again once
c.mu.Lock()
c.ticket = ""
c.mu.Unlock()
if err := c.Login(ctx); err != nil {
return err
}
err = c.send(ctx, method, path, query, form, out, true)
}
return err
}
func (c *Client) send(ctx context.Context, method, path string, query, form url.Values, out any, auth bool) error {
u := *c.base
u.Path = "/api2/json/" + strings.TrimLeft(path, "/")
if len(query) > 0 {
@@ -166,14 +296,21 @@ func (c *Client) do(ctx context.Context, method, path string, query, form url.Va
if err != nil {
return err
}
req.Header.Set("Authorization", c.auth)
if auth {
c.mu.Lock()
req.AddCookie(&http.Cookie{Name: "PVEAuthCookie", Value: c.ticket})
if method != http.MethodGet {
req.Header.Set("CSRFPreventionToken", c.csrf)
}
c.mu.Unlock()
}
req.Header.Set("Accept", "application/json")
if form != nil {
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
}
resp, err := c.http.Do(req)
if err != nil {
// url.Error contains the URL only, never the Authorization header.
// url.Error contains the URL only, never the ticket nor the password.
return fmt.Errorf("cluster %s: %w", c.name, err)
}
defer resp.Body.Close()
internal/pve/pvetest/access.go
+187
View File
@@ -0,0 +1,187 @@
package pvetest
import (
"net/http"
"sort"
"strings"
)
// access fakes /access/groups, /access/roles and /access/users (user.cfg).
func (s *Server) access(w http.ResponseWriter, r *http.Request, method string, seg []string) {
f := r.PostForm
if method == http.MethodDelete {
f = r.URL.Query()
}
switch {
case len(seg) == 2 && seg[1] == "groups" && method == "GET":
var out []any
for _, id := range sortedKeys(s.Groups) {
o := map[string]any{"groupid": id, "users": strings.Join(s.members(id), ",")}
for k, v := range s.Groups[id] {
o[k] = v
}
out = append(out, o)
}
reply(w, out)
case len(seg) == 2 && seg[1] == "groups" && method == "POST":
id := f.Get("groupid")
if s.Groups[id] != nil {
fail(w, 500, "create group failed: group '"+id+"' already exists")
return
}
s.Groups[id] = setFields(map[string]any{}, f, "groupid")
reply(w, nil)
case len(seg) == 3 && seg[1] == "groups":
g := s.Groups[seg[2]]
if g == nil {
fail(w, 500, "group '"+seg[2]+"' does not exist")
return
}
switch method {
case "PUT":
setFields(g, f)
case "DELETE":
delete(s.Groups, seg[2])
for _, u := range s.Users {
u["groups"] = strings.Join(remove(split(u["groups"]), seg[2]), ",")
}
}
reply(w, nil)
case len(seg) == 2 && seg[1] == "roles" && method == "GET":
out := []any{
map[string]any{"roleid": "Administrator", "privs": "Sys.Audit,Sys.Modify", "special": 1},
map[string]any{"roleid": "PVEAuditor", "privs": "Sys.Audit", "special": 1},
}
for _, id := range sortedKeys(s.Roles) {
out = append(out, map[string]any{"roleid": id, "privs": s.Roles[id], "special": 0})
}
reply(w, out)
case len(seg) == 2 && seg[1] == "roles" && method == "POST":
s.Roles[f.Get("roleid")] = f.Get("privs")
reply(w, nil)
case len(seg) == 3 && seg[1] == "roles":
if _, ok := s.Roles[seg[2]]; !ok {
fail(w, 500, "role '"+seg[2]+"' does not exist")
return
}
if method == "DELETE" {
delete(s.Roles, seg[2])
} else {
s.Roles[seg[2]] = f.Get("privs")
}
reply(w, nil)
case len(seg) == 2 && seg[1] == "users" && method == "GET":
var out []any
for _, id := range sortedKeys(s.Users) {
realm := id[strings.LastIndex(id, "@")+1:]
o := map[string]any{"userid": id, "realm-type": realm, "tokens": []any{}}
for k, v := range s.Users[id] {
o[k] = v
}
out = append(out, o)
}
reply(w, out)
case len(seg) == 2 && seg[1] == "users" && method == "POST":
id := f.Get("userid")
if s.Users[id] != nil {
fail(w, 500, "create user failed: user '"+id+"' already exists")
return
}
if !s.groupsExist(w, f.Get("groups")) {
return
}
s.Users[id] = setFields(map[string]any{"enable": 1}, f, "userid", "password")
reply(w, nil)
case len(seg) == 3 && seg[1] == "users":
u := s.Users[seg[2]]
if u == nil {
fail(w, 500, "user '"+seg[2]+"' does not exist")
return
}
switch method {
case "PUT":
if !s.groupsExist(w, f.Get("groups")) {
return
}
setFields(u, f)
case "DELETE":
delete(s.Users, seg[2])
delete(s.Shadow, strings.TrimSuffix(seg[2], "@pve"))
}
reply(w, nil)
default:
fail(w, 501, "not implemented in fake: "+method+" "+strings.Join(seg, "/"))
}
}
func (s *Server) groupsExist(w http.ResponseWriter, groups string) bool {
for _, g := range split(groups) {
if s.Groups[g] == nil {
fail(w, 500, "no such group '"+g+"'")
return false
}
}
return true
}
func (s *Server) members(group string) []string {
var out []string
for id, u := range s.Users {
for _, g := range split(u["groups"]) {
if g == group {
out = append(out, id)
}
}
}
sort.Strings(out)
return out
}
// setFields copies form values; an empty value removes the field (like user.cfg).
func setFields(m map[string]any, f map[string][]string, skip ...string) map[string]any {
sk := map[string]bool{}
for _, k := range skip {
sk[k] = true
}
for k, v := range f {
if sk[k] || len(v) == 0 {
continue
}
if v[0] == "" {
delete(m, k)
continue
}
m[k] = v[0]
}
return m
}
func split(v any) []string {
s, _ := v.(string)
var out []string
for _, x := range strings.Split(s, ",") {
if x = strings.TrimSpace(x); x != "" {
out = append(out, x)
}
}
return out
}
func remove(l []string, x string) []string {
out := l[:0]
for _, e := range l {
if e != x {
out = append(out, e)
}
}
return out
}
func sortedKeys[V any](m map[string]V) []string {
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
internal/pve/pvetest/fake.go
+65 -17
View File
@@ -1,7 +1,9 @@
// Package pvetest is an in-memory fake of the subset of the PVE 9.x API used by
// pve-confsync, for tests. It mimics the behaviours that matter for the engine:
// parsed datacenter options on GET, property strings on PUT, SDN pending/running
// configurations, global SDN lock with token, rollback and reload task.
// pve-confsync, for tests and the demo. It mimics the behaviours that matter for
// the engine: ticket login of root@pam, parsed datacenter options on GET,
// property strings on PUT, SDN pending/running configurations, global SDN lock
// with token, rollback and reload task, groups, roles and accounts of user.cfg,
// and an SSH server serving /etc/pve/priv/shadow.cfg (see ssh.go).
package pvetest
import (
@@ -43,29 +45,51 @@ func (s SDN) clone() SDN {
return c
}
// Password of root@pam on the fake clusters (API and SSH).
const Password = "root-secret"
// Server is a fake PVE cluster.
type Server struct {
*httptest.Server
mu sync.Mutex
Version string // pve-manager version, e.g. 9.1.4
Password string // root@pam password
Options map[string]string // datacenter.cfg as written (property strings)
Pending SDN
Running SDN
Lock string
Perms map[string]map[string]any
Groups map[string]map[string]any // groupid -> fields (comment)
Roles map[string]string // custom roles: roleid -> privs
Users map[string]map[string]any // userid -> fields (groups as "a,b")
Shadow map[string]string // user without realm -> hash (priv/shadow.cfg)
FailOn string // "METHOD path" answering 500, to test rollbacks
Calls []string // "METHOD path" of every write
Calls []string // "METHOD path" of every write, "SSH write" for shadow.cfg
Nodes []*Node // nodes of the node replication (AddNode); none: one node "n1"
locks int
ssh *sshServer
}
// New starts a fake cluster.
const ticket, csrf = "PVE:root@pam:FAKE::sig", "fake-csrf"
// New starts a fake cluster (API in HTTPS and SSH).
func New(version string) *Server {
s := &Server{Version: version, Options: map[string]string{}, Pending: newSDN(), Running: newSDN(),
Perms: map[string]map[string]any{}}
s := &Server{Version: version, Password: Password, Options: map[string]string{}, Pending: newSDN(), Running: newSDN(),
Groups: map[string]map[string]any{}, Roles: map[string]string{}, Shadow: map[string]string{},
Users: map[string]map[string]any{"root@pam": {"enable": 1, "email": "root@localhost"}}}
s.Server = httptest.NewTLSServer(http.HandlerFunc(s.handle))
s.ssh = startSSH(s)
return s
}
// Close stops the API and the SSH server.
func (s *Server) Close() {
s.ssh.close()
for _, n := range s.Nodes {
n.ssh.close()
}
s.Server.Close()
}
// Commit makes the pending SDN configuration the running one (like an apply).
func (s *Server) Commit() {
s.mu.Lock()
@@ -87,12 +111,24 @@ func fail(w http.ResponseWriter, code int, msg string) {
func (s *Server) handle(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
defer s.mu.Unlock()
if !strings.HasPrefix(r.Header.Get("Authorization"), "PVEAPIToken=") {
_ = r.ParseForm()
p := strings.TrimPrefix(r.URL.Path, "/api2/json/")
if r.Method == http.MethodPost && p == "access/ticket" {
if r.PostForm.Get("username") != "root@pam" || r.PostForm.Get("password") != s.Password {
fail(w, 401, "authentication failure")
return
}
reply(w, map[string]any{"ticket": ticket, "CSRFPreventionToken": csrf, "username": "root@pam"})
return
}
if c, err := r.Cookie("PVEAuthCookie"); err != nil || c.Value != ticket {
fail(w, 401, "no ticket")
return
}
_ = r.ParseForm()
p := strings.TrimPrefix(r.URL.Path, "/api2/json/")
if r.Method != http.MethodGet && r.Header.Get("CSRFPreventionToken") != csrf {
fail(w, 401, "Permission denied - invalid csrf token")
return
}
seg := strings.Split(p, "/")
key := r.Method + " " + p
if r.Method != http.MethodGet {
@@ -114,14 +150,26 @@ func (s *Server) handle(w http.ResponseWriter, r *http.Request) {
if running {
cfg = &s.Running
}
var nd *Node
if len(seg) >= 3 && seg[0] == "nodes" {
nd = s.node(seg[1])
}
switch {
case nd != nil && r.Method == "GET" && len(seg) == 3 && seg[2] == "version":
reply(w, map[string]any{"version": nd.version(), "release": release(nd.version()), "repoid": "abcdef"})
case nd != nil && (seg[2] == "network" || seg[2] == "dns"):
s.nodeAPI(w, r, nd, seg)
case key == "GET version" || (r.Method == "GET" && len(seg) == 3 && seg[0] == "nodes" && seg[2] == "version"):
reply(w, map[string]any{"version": s.Version, "release": release(s.Version), "repoid": "abcdef"})
case key == "GET cluster/status":
reply(w, []any{
map[string]any{"type": "cluster", "name": "fake", "quorate": 1},
map[string]any{"type": "node", "name": "n1", "online": 1},
})
st := []any{map[string]any{"type": "cluster", "name": "fake", "quorate": 1}}
if len(s.Nodes) == 0 {
st = append(st, map[string]any{"type": "node", "name": "n1", "online": 1})
}
for i, n := range s.Nodes {
st = append(st, map[string]any{"type": "node", "name": n.Name, "online": 1, "nodeid": i + 1, "ip": "127.0.0.1"})
}
reply(w, st)
case r.Method == "GET" && len(seg) == 4 && seg[0] == "nodes" && seg[2] == "sdn" && seg[3] == "zones":
var out []any
for id := range s.Running.Zones {
@@ -130,8 +178,8 @@ func (s *Server) handle(w http.ResponseWriter, r *http.Request) {
reply(w, out)
case r.Method == "GET" && len(seg) == 5 && seg[0] == "nodes" && seg[2] == "tasks" && seg[4] == "status":
reply(w, map[string]any{"status": "stopped", "exitstatus": "OK"})
case key == "GET access/permissions":
reply(w, s.Perms)
case strings.HasPrefix(p, "access/"):
s.access(w, r, r.Method, seg)
case key == "GET cluster/options":
reply(w, s.parsedOptions())
case key == "PUT cluster/options":
internal/pve/pvetest/nodes.go
+419
View File
@@ -0,0 +1,419 @@
package pvetest
import (
"bufio"
"bytes"
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"fmt"
"io"
"net"
"net/http"
"path"
"sort"
"strings"
"golang.org/x/crypto/ssh"
)
// FakeFile is a file of a fake node.
type FakeFile struct {
Content string
Mode string // 644 by default
Owner string // root:root by default
}
// Node is a node of a fake cluster for the node replication: its own SSH
// server (files, reloads, restore timer), and its network interfaces and DNS
// servers served by the API of the cluster (/nodes/{node}/network, /dns).
// Like PVE, the interface writes go to a pending configuration
// (interfaces.new) applied by PUT /nodes/{node}/network.
type Node struct {
Name string
Hostname string // answered by hostname (default: Name)
Version string // pve-manager version (default: the cluster's)
Files map[string]FakeFile
Net map[string]map[string]any // iface -> fields as written (cidr, type…)
Pending map[string]map[string]any // interfaces.new; nil: no pending change
DNS map[string]string
Reloads []string // reload commands run, "network" for PUT /network
Armed map[string]bool // restore timers armed (systemd-run units)
FailOn string // "reload <cmd>", "prepare" or "probe": that step fails
Calls []string // SSH steps run
backups map[string]backup
srv *Server
ssh *sshServer
}
type backup struct {
files map[string]*FakeFile // nil: absent before
net map[string]map[string]any
pending map[string]map[string]any
dns map[string]string
hasNet bool
hasDNS bool
}
// AddNode adds a node to the fake cluster (listed by GET /cluster/status).
func (s *Server) AddNode(name string) *Node {
n := &Node{Name: name, Hostname: name, Files: map[string]FakeFile{}, Net: map[string]map[string]any{},
DNS: map[string]string{}, Armed: map[string]bool{}, backups: map[string]backup{}, srv: s}
n.ssh = newSSH(func(user, pass string) bool {
s.mu.Lock()
defer s.mu.Unlock()
return user == "root" && pass == s.Password
}, n.exec)
s.mu.Lock()
s.Nodes = append(s.Nodes, n)
s.mu.Unlock()
return n
}
// SSHAddr returns the host and port of the SSH server of the node.
func (n *Node) SSHAddr() (string, int) {
a := n.ssh.ln.Addr().(*net.TCPAddr)
return a.IP.String(), a.Port
}
// SSHHostKey returns the host key of the node (authorized_keys format).
func (n *Node) SSHHostKey() string {
return strings.TrimSpace(string(ssh.MarshalAuthorizedKey(n.ssh.key)))
}
func (s *Server) node(name string) *Node {
for _, n := range s.Nodes {
if n.Name == name {
return n
}
}
return nil
}
func (n *Node) version() string {
if n.Version != "" {
return n.Version
}
return n.srv.Version
}
func cloneNet(m map[string]map[string]any) map[string]map[string]any {
if m == nil {
return nil
}
out := map[string]map[string]any{}
for k, v := range m {
f := map[string]any{}
for x, y := range v {
f[x] = y
}
out[k] = f
}
return out
}
// ---- API (called with s.mu held) ----
func (s *Server) nodeAPI(w http.ResponseWriter, r *http.Request, n *Node, seg []string) {
switch {
case len(seg) == 3 && seg[2] == "dns" && r.Method == "GET":
out := map[string]any{}
for k, v := range n.DNS {
out[k] = v
}
reply(w, out)
case len(seg) == 3 && seg[2] == "dns" && r.Method == "PUT":
if r.PostForm.Get("search") == "" {
fail(w, 400, "search: property is missing and it is not optional")
return
}
n.DNS = map[string]string{}
for _, k := range []string{"search", "dns1", "dns2", "dns3"} {
if v := r.PostForm.Get(k); v != "" {
n.DNS[k] = v
}
}
reply(w, nil)
case len(seg) == 3 && seg[2] == "network" && r.Method == "GET":
cfg := n.Net
if n.Pending != nil {
cfg = n.Pending
}
var out []any
prio := 0
for _, id := range sortedKeys(cfg) {
prio++
f := map[string]any{"iface": id, "priority": prio, "families": []any{"inet"}, "method": "manual", "active": 1}
for k, v := range cfg[id] {
f[k] = v
}
if c, ok := f["cidr"].(string); ok {
a, m, _ := strings.Cut(c, "/")
f["address"], f["netmask"], f["method"] = a, m, "static"
}
if f["type"] == "eth" {
f["exists"] = 1
}
out = append(out, f)
}
reply(w, out)
case len(seg) == 3 && seg[2] == "network" && r.Method == "DELETE":
n.Pending = nil
reply(w, nil)
case len(seg) == 3 && seg[2] == "network" && r.Method == "PUT":
if n.Pending != nil {
n.Net, n.Pending = n.Pending, nil
}
n.Reloads = append(n.Reloads, "network")
reply(w, "UPID:"+n.Name+":00001234:00005678:65000000:srvreload:networking:root@pam:")
case len(seg) == 3 && seg[2] == "network" && r.Method == "POST":
id := r.PostForm.Get("iface")
cfg := n.pending()
if cfg[id] != nil {
fail(w, 400, "iface: interface already exists")
return
}
cfg[id] = fields(r.PostForm, "iface")
reply(w, nil)
case len(seg) == 4 && seg[2] == "network":
cfg := n.pending()
cur := cfg[seg[3]]
if cur == nil {
fail(w, 400, "iface: interface does not exist")
return
}
switch r.Method {
case "DELETE":
delete(cfg, seg[3])
case "PUT":
if r.PostForm.Get("type") == "" {
fail(w, 400, "type: property is missing and it is not optional")
return
}
for _, d := range strings.Split(r.PostForm.Get("delete"), ",") {
delete(cur, d)
}
// like PVE: without an address in the parameters, the interface
// becomes "manual" and loses its address
if r.PostForm.Get("cidr") == "" {
delete(cur, "cidr")
}
for k, v := range fields(r.PostForm, "delete") {
cur[k] = v
}
default:
fail(w, 501, "not implemented in fake")
return
}
reply(w, nil)
default:
fail(w, 501, "not implemented in fake: "+r.Method+" "+strings.Join(seg, "/"))
}
}
// pending returns interfaces.new, created from the current configuration.
func (n *Node) pending() map[string]map[string]any {
if n.Pending == nil {
n.Pending = cloneNet(n.Net)
}
return n.Pending
}
// ---- SSH ----
func (n *Node) exec(cmd string, in io.Reader, out, errw io.Writer) int {
s := n.srv
s.mu.Lock()
defer s.mu.Unlock()
first, _, _ := strings.Cut(cmd, "\n")
verb, arg, _ := strings.Cut(strings.TrimPrefix(first, "# confsync:"), " ")
if strings.HasPrefix(first, "# confsync:") {
n.Calls = append(n.Calls, verb)
if n.FailOn == verb || n.FailOn == verb+" "+arg {
fmt.Fprintln(errw, "injected failure")
return 1
}
}
switch {
case strings.HasPrefix(cmd, "pveversion"):
fmt.Fprintf(out, "pve-manager/%s/abcdef (running kernel: 6.14.8-2-pve)\n", n.version())
if strings.Contains(cmd, "hostname") {
fmt.Fprintln(out, n.Hostname)
}
return 0
case verb == "read":
return n.read(cmd, out)
case verb == "prepare":
data, _ := io.ReadAll(in)
return n.prepare(arg, data, errw)
case verb == "restore":
b, ok := n.backups[arg]
if !ok {
fmt.Fprintln(errw, "no backup "+arg)
return 1
}
n.restore(b)
return 0
case verb == "arm":
n.Armed[arg] = true
return 0
case verb == "disarm":
delete(n.Armed, arg)
return 0
case verb == "reload", verb == "probe":
if verb == "reload" {
n.Reloads = append(n.Reloads, arg)
}
return 0
}
fmt.Fprintf(errw, "fake node: unknown command %q\n", first)
return 127
}
// Fire runs the armed restore timers (as systemd would after the delay).
func (n *Node) Fire() {
n.srv.mu.Lock()
defer n.srv.mu.Unlock()
for unit := range n.Armed {
dir := strings.Replace(unit, "pve-confsync-restore-", "/var/lib/pve-confsync/backup/", 1)
if b, ok := n.backups[dir]; ok {
n.restore(b)
}
n.Reloads = append(n.Reloads, "ifreload -a")
delete(n.Armed, unit)
}
}
func (n *Node) read(cmd string, out io.Writer) int {
var pats []string
for _, l := range strings.Split(cmd, "\n") {
if strings.HasPrefix(l, "for f in ") {
pats = strings.Fields(strings.TrimSuffix(strings.TrimPrefix(l, "for f in "), "; do"))
}
}
for _, p := range sortedKeys(n.Files) {
for _, pat := range pats {
if ok, _ := path.Match(pat, p); ok {
f := n.Files[p]
fmt.Fprintf(out, "F %s %s %s\n%s\n", def(f.Mode, "644"), def(f.Owner, "root:root"), p,
base64.StdEncoding.EncodeToString([]byte(f.Content)))
break
}
}
}
if n.Pending != nil {
fmt.Fprintln(out, "PENDING")
}
return 0
}
func def(v, d string) string {
if v == "" {
return d
}
return v
}
// prepare emulates the prepare script: checks, backups, writes.
func (n *Node) prepare(dir string, plan []byte, errw io.Writer) int {
type op struct{ op, mode, owner, hash, path, data string }
var ops []op
sc := bufio.NewScanner(bytes.NewReader(plan))
sc.Buffer(make([]byte, 64<<10), 8<<20)
for sc.Scan() {
p := strings.SplitN(sc.Text(), " ", 5)
if len(p) != 5 {
continue
}
o := op{p[0], p[1], p[2], p[3], p[4], ""}
if o.op == "W" && sc.Scan() {
o.data = sc.Text()
}
ops = append(ops, o)
}
for _, o := range ops {
if o.op != "W" && o.op != "D" {
continue
}
h := "absent"
if f, ok := n.Files[o.path]; ok {
h = sha([]byte(f.Content))
}
if h != o.hash {
fmt.Fprintf(errw, "changed since the dry-run: %s\n", o.path)
return 3
}
}
b := backup{files: map[string]*FakeFile{}}
for _, o := range ops {
switch o.path {
case "/etc/network/interfaces", "/etc/network/interfaces.new":
b.hasNet, b.net, b.pending = true, cloneNet(n.Net), cloneNet(n.Pending)
continue
case "/etc/resolv.conf":
b.hasDNS, b.dns = true, map[string]string{}
for k, v := range n.DNS {
b.dns[k] = v
}
continue
}
if f, ok := n.Files[o.path]; ok {
c := f
b.files[o.path] = &c
} else {
b.files[o.path] = nil
}
}
n.backups[dir] = b
for _, o := range ops {
switch o.op {
case "W":
data, err := base64.StdEncoding.DecodeString(o.data)
if err != nil {
n.restore(b)
fmt.Fprintln(errw, "write failed: "+o.path)
return 4
}
n.Files[o.path] = FakeFile{Content: string(data), Mode: o.mode, Owner: o.owner}
case "D":
delete(n.Files, o.path)
}
}
return 0
}
func (n *Node) restore(b backup) {
for p, f := range b.files {
if f == nil {
delete(n.Files, p)
} else {
n.Files[p] = *f
}
}
if b.hasNet {
n.Net, n.Pending = cloneNet(b.net), cloneNet(b.pending)
}
if b.hasDNS {
n.DNS = map[string]string{}
for k, v := range b.dns {
n.DNS[k] = v
}
}
}
// Backups lists the backup folders written on the node.
func (n *Node) Backups() []string {
n.srv.mu.Lock()
defer n.srv.mu.Unlock()
var out []string
for d := range n.backups {
out = append(out, d)
}
sort.Strings(out)
return out
}
func sha(b []byte) string {
h := sha256.Sum256(b)
return hex.EncodeToString(h[:])
}
internal/pve/pvetest/ssh.go
+152
View File
@@ -0,0 +1,152 @@
package pvetest
import (
"bufio"
"bytes"
"crypto/ed25519"
"crypto/rand"
"encoding/binary"
"fmt"
"io"
"net"
"strings"
"golang.org/x/crypto/ssh"
)
// sshServer answers the commands pve-confsync runs on a node: pveversion, the
// read of priv/shadow.cfg and its locked write (adapter password).
type sshServer struct {
ln net.Listener
key ssh.PublicKey
exec func(cmd string, in io.Reader, out, errw io.Writer) int
}
func startSSH(s *Server) *sshServer {
return newSSH(func(user, pass string) bool {
s.mu.Lock()
defer s.mu.Unlock()
return user == "root" && pass == s.Password
}, s.exec)
}
// newSSH starts an SSH server accepting the password check ok and running
// the commands with exec.
func newSSH(ok func(user, pass string) bool, exec func(cmd string, in io.Reader, out, errw io.Writer) int) *sshServer {
_, priv, _ := ed25519.GenerateKey(rand.Reader)
signer, _ := ssh.NewSignerFromKey(priv)
cfg := &ssh.ServerConfig{
PasswordCallback: func(c ssh.ConnMetadata, pass []byte) (*ssh.Permissions, error) {
if ok(c.User(), string(pass)) {
return nil, nil
}
return nil, fmt.Errorf("denied")
},
}
cfg.AddHostKey(signer)
ln, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
panic(err)
}
srv := &sshServer{ln: ln, key: signer.PublicKey(), exec: exec}
go func() {
for {
nc, err := ln.Accept()
if err != nil {
return
}
go srv.serve(nc, cfg)
}
}()
return srv
}
func (x *sshServer) close() { x.ln.Close() }
// SSHAddr returns the host and port of the fake SSH server.
func (s *Server) SSHAddr() (string, int) {
a := s.ssh.ln.Addr().(*net.TCPAddr)
return a.IP.String(), a.Port
}
// SSHHostKey returns the host key of the fake node (authorized_keys format).
func (s *Server) SSHHostKey() string {
return strings.TrimSpace(string(ssh.MarshalAuthorizedKey(s.ssh.key)))
}
func (x *sshServer) serve(nc net.Conn, cfg *ssh.ServerConfig) {
_, chans, reqs, err := ssh.NewServerConn(nc, cfg)
if err != nil {
nc.Close()
return
}
go ssh.DiscardRequests(reqs)
for nch := range chans {
if nch.ChannelType() != "session" {
_ = nch.Reject(ssh.UnknownChannelType, "session only")
continue
}
ch, reqs, err := nch.Accept()
if err != nil {
continue
}
go func() {
defer ch.Close()
for req := range reqs {
if req.Type != "exec" {
_ = req.Reply(false, nil)
continue
}
var p struct{ Cmd string }
_ = ssh.Unmarshal(req.Payload, &p)
_ = req.Reply(true, nil)
status := x.exec(p.Cmd, ch, ch, ch.Stderr())
b := make([]byte, 4)
binary.BigEndian.PutUint32(b, uint32(status))
_, _ = ch.SendRequest("exit-status", false, b)
return
}
}()
}
}
func (s *Server) exec(cmd string, in io.Reader, out, errw io.Writer) int {
switch {
case strings.HasPrefix(cmd, "pveversion"):
fmt.Fprintf(out, "pve-manager/%s/abcdef (running kernel: 6.14.8-2-pve)\n", s.Version)
return 0
case strings.HasPrefix(cmd, "cat /etc/pve/priv/shadow.cfg"):
s.mu.Lock()
defer s.mu.Unlock()
for _, u := range sortedKeys(s.Shadow) {
fmt.Fprintf(out, "%s:%s:\n", u, s.Shadow[u])
}
return 0
case strings.Contains(cmd, "confsync-shadow-write"):
data, _ := io.ReadAll(in)
s.mu.Lock()
defer s.mu.Unlock()
s.Calls = append(s.Calls, "SSH write shadow.cfg")
if s.FailOn == "SSH write shadow.cfg" {
fmt.Fprintln(errw, "injected failure")
return 1
}
sc := bufio.NewScanner(bytes.NewReader(data))
for sc.Scan() {
p := strings.SplitN(sc.Text(), "\t", 3)
switch {
case len(p) == 3 && p[0] == "set":
s.Shadow[p[1]] = p[2]
case len(p) >= 2 && p[0] == "del":
delete(s.Shadow, p[1])
default:
fmt.Fprintln(errw, "invalid operation")
return 1
}
}
fmt.Fprintln(out, "ok")
return 0
}
fmt.Fprintf(errw, "fake: unknown command %q\n", cmd)
return 127
}
internal/server/api.go
+164 -135
View File
@@ -7,13 +7,13 @@ import (
"fmt"
"log"
"net/http"
"net/url"
"strconv"
"strings"
"sync"
"time"
"gopkg.in/yaml.v3"
"pve-confsync/internal/adapter"
"pve-confsync/internal/config"
"pve-confsync/internal/engine"
"pve-confsync/internal/model"
@@ -33,7 +33,13 @@ func (s *Server) handleState(w http.ResponseWriter, r *http.Request) {
if s.state == nil || r.URL.Query().Get("refresh") == "1" {
cfg, err := s.store.Config(r.Context())
if errors.Is(err, store.ErrNoReference) {
writeJSON(w, http.StatusOK, map[string]any{"setup": true})
// nothing to compare yet: no cluster at all, or no primary chosen
cls, _ := s.store.Clusters(r.Context())
setup := "clusters"
if len(cls) > 0 {
setup = "primary"
}
writeJSON(w, http.StatusOK, map[string]any{"setup": setup})
return
}
if err != nil {
@@ -78,23 +84,29 @@ func (s *Server) remember(st *service.State, all bool) {
// ---- dry-run and apply (phase 2) ----
type selection struct {
Scope string `json:"scope"` // global, client or object
Scope string `json:"scope"` // global (several targets), client (one target) or object
Targets []string `json:"targets"` // global scope: the targets (empty: all of them)
Client string `json:"client"` // client and object scopes
Objects []string `json:"objects"` // object scope: "kind/id"
Skip []model.Kind `json:"skip"` // kinds left out of this run ("not synchronized")
Digests map[string]string `json:"digests"` // apply: digest of each reviewed plan
Confirm string `json:"confirm"` // apply with SDN changes: client name (reference name for global)
Confirm string `json:"confirm"` // apply with SDN changes: target name (primary name for global)
}
func (sel *selection) check() error {
if len(sel.Skip) > 0 {
sel.Skip = adapter.Bundle(sel.Skip)
}
switch sel.Scope {
case "global":
sel.Client, sel.Objects = "", nil
case "client":
sel.Objects = nil
sel.Objects, sel.Targets = nil, nil
if sel.Client == "" {
return errors.New("client required")
}
case "object":
sel.Targets = nil
if sel.Client == "" || len(sel.Objects) == 0 {
return errors.New("client and objects required")
}
@@ -106,27 +118,30 @@ func (sel *selection) check() error {
func (sel selection) clients() []string {
if sel.Client == "" {
return nil
return sel.Targets
}
return []string{sel.Client}
}
// all tells whether the selection reads every target (the cached comparison is replaced).
func (sel selection) all() bool { return sel.Scope == "global" && len(sel.Targets) == 0 }
// plan reads the clusters and returns the plans of the selection.
func (s *Server) plan(ctx context.Context, sel selection) (*config.Config, *service.State, []service.ClientPlan, error) {
cfg, err := s.store.Config(ctx)
if err != nil {
return nil, nil, nil, err
}
if sel.Client != "" {
if _, ok := cfg.Client(sel.Client); !ok {
return nil, nil, nil, fmt.Errorf("unknown client %q", sel.Client)
for _, c := range sel.clients() {
if _, ok := cfg.Client(c); !ok {
return nil, nil, nil, fmt.Errorf("unknown target %q", c)
}
}
rctx, cancel := context.WithTimeout(ctx, readTimeout)
defer cancel()
st := service.Analyze(rctx, cfg, sel.clients())
s.remember(st, sel.Scope == "global")
return cfg, st, st.Plans(sel.Objects), nil
s.remember(st, sel.all())
return cfg, st, st.Plans(sel.Objects, sel.Skip), nil
}
type runDetail struct {
@@ -179,13 +194,25 @@ func planClients(plans []service.ClientPlan) []string {
}
func target(sel selection) string {
switch sel.Scope {
case "client":
return sel.Client
case "object":
return sel.Client + ": " + strings.Join(sel.Objects, ", ")
var t string
switch {
case sel.Scope == "client":
t = sel.Client
case sel.Scope == "object":
t = sel.Client + ": " + strings.Join(sel.Objects, ", ")
case len(sel.Targets) > 0:
t = "targets " + strings.Join(sel.Targets, ", ")
default:
t = "all targets"
}
return "global"
if len(sel.Skip) > 0 {
var ks []string
for _, k := range sel.Skip {
ks = append(ks, string(k))
}
t += " (not replicated: " + strings.Join(ks, ", ") + ")"
}
return t
}
// dryRunSummary counts the changes: "3 changes on 2 clients", or the errors.
@@ -203,11 +230,11 @@ func dryRunSummary(plans []service.ClientPlan) (string, string) {
}
switch {
case blocked > 0:
return store.RunBlocked, fmt.Sprintf("%d changement(s) sur %d client(s), %d client(s) en erreur", changes, clients, blocked)
return store.RunBlocked, fmt.Sprintf("%d change(s) on %d target(s), %d target(s) in error", changes, clients, blocked)
case changes == 0:
return store.RunNoop, "aucun écart"
return store.RunNoop, "no difference"
}
return store.RunOK, fmt.Sprintf("%d changement(s) sur %d client(s)", changes, clients)
return store.RunOK, fmt.Sprintf("%d change(s) on %d target(s)", changes, clients)
}
// handleApply recomputes the plans, checks that they are the ones reviewed in the
@@ -225,7 +252,7 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
}
id := identity(r)
if !s.applyMu.TryLock() {
writeError(w, http.StatusConflict, "une synchronisation est déjà en cours", "busy")
writeError(w, http.StatusConflict, "a replication is already running", "busy")
return
}
started := false
@@ -252,8 +279,8 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusConflict, p.Client+": "+strings.Join(p.Plan.Errors, "; "), "plan")
return
case sel.Digests[p.Client] != p.Digest:
s.audit(r, id, "apply", target(sel), false, p.Client+": l'état a changé depuis le dry-run")
writeJSON(w, http.StatusConflict, map[string]any{"error": p.Client + ": l'état a changé depuis le dry-run, relancer le dry-run",
s.audit(r, id, "apply", target(sel), false, p.Client+": the state changed since the dry-run")
writeJSON(w, http.StatusConflict, map[string]any{"error": p.Client + ": the state changed since the dry-run, run it again",
"code": "stale", "plans": plans})
return
case p.Plan.Empty():
@@ -263,7 +290,7 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
todo = append(todo, p)
}
if len(todo) == 0 {
writeJSON(w, http.StatusOK, map[string]any{"run": 0, "summary": "aucun écart"})
writeJSON(w, http.StatusOK, map[string]any{"run": 0, "summary": "no difference"})
return
}
if sdn {
@@ -272,7 +299,7 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
want = cfg.Reference.Name
}
if sel.Confirm != want {
writeError(w, http.StatusBadRequest, "changements SDN : confirmation requise ("+want+")", "confirm")
writeError(w, http.StatusBadRequest, "SDN changes: confirmation required ("+want+")", "confirm")
return
}
}
@@ -319,7 +346,7 @@ func (s *Server) runApply(r *http.Request, id Identity, runID int64, sel selecti
res := det.Results[p.Client]
res.Status = store.RunRunning
save(store.RunRunning, fmt.Sprintf("%s en cours", p.Client))
err := st.Apply(ctx, p, sel.Objects, func(step engine.Step) {
err := st.Apply(ctx, p, sel.Objects, sel.Skip, func(step engine.Step) {
mu.Lock()
res.Steps = append(res.Steps, step)
mu.Unlock()
@@ -333,14 +360,14 @@ func (s *Server) runApply(r *http.Request, id Identity, runID int64, sel selecti
} else {
done++
res.Status = store.RunOK
s.audit(r, id, "apply", p.Client, true, fmt.Sprintf("%d changement(s), vérifié", len(p.Plan.Changes)))
s.audit(r, id, "apply", p.Client, true, fmt.Sprintf("%d change(s), verified", len(p.Plan.Changes)))
}
}
status := store.RunOK
summary := fmt.Sprintf("%d client(s) synchronisé(s) et vérifié(s)", done)
summary := fmt.Sprintf("%d target(s) replicated and verified", done)
if failed > 0 {
status = store.RunError
summary = fmt.Sprintf("%d client(s) en échec, %d synchronisé(s)", failed, done)
summary = fmt.Sprintf("%d target(s) failed, %d replicated", failed, done)
}
save(status, summary)
s.invalidate()
@@ -387,8 +414,8 @@ func (s *Server) handleClusters(w http.ResponseWriter, r *http.Request) {
}
// connection details are for the administrators
if identity(r).Role != store.RoleAdmin {
for i := range cls {
cls[i] = store.Cluster{Name: cls[i].Name, Reference: cls[i].Reference, URL: cls[i].URL, Mapping: cls[i].Mapping, Policies: cls[i].Policies}
for i, c := range cls {
cls[i] = store.Cluster{Name: c.Name, Role: c.Role, URL: c.URL, Mapping: c.Mapping, Policies: c.Policies, LastCheck: c.LastCheck}
}
}
writeJSON(w, http.StatusOK, map[string]any{"clusters": cls})
@@ -409,12 +436,12 @@ func (s *Server) handleSaveCluster(w http.ResponseWriter, r *http.Request) {
return
}
err := s.store.SaveCluster(r.Context(), old, c)
detail := fmt.Sprintf("url=%s token=%s reference=%t", c.URL, c.TokenID, c.Reference)
if c.Secret != "" {
detail += " secret=modifié"
detail := fmt.Sprintf("url=%s role=%s tls=%s ssh=%s:%d", c.URL, c.Role, c.TLS, c.SSHHost, c.SSHPort)
if c.Password != "" {
detail += " password=changed"
}
if old != "" && old != c.Name {
detail += " renommé depuis " + old
detail += " renamed from " + old
}
s.audit(r, Identity{}, "cluster-save", c.Name, err == nil, errText(err, detail))
if err != nil {
@@ -425,19 +452,25 @@ func (s *Server) handleSaveCluster(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// validateCluster checks the mapping and policies of a cluster with the rules of
// the YAML configuration.
// validateCluster checks the address and the mapping and policies of a cluster.
func validateCluster(c store.Cluster) error {
if c.Fingerprint == "" && c.CAPEM == "" && !c.Insecure && !strings.HasPrefix(c.URL, "https://") {
return errors.New("url: https:// expected")
u, err := url.Parse(strings.TrimSpace(c.URL))
if err != nil || u.Scheme != "https" || u.Host == "" {
return errors.New("url: https://host:8006 expected")
}
if c.User != "" && c.User != "root@pam" {
return errors.New("user: root@pam expected (the same account is used over SSH)")
}
if c.Timeout != "" {
if _, err := time.ParseDuration(c.Timeout); err != nil {
return fmt.Errorf("timeout: %w", err)
}
}
cfg := &config.Config{Kinds: model.Kinds, Reference: config.Cluster{Name: "\x00ref", Secret: "x"},
Clients: []config.Client{{Cluster: config.Cluster{Name: c.Name, Secret: "x"}, Mapping: c.Mapping, Policies: c.Policies}}}
if c.SSHPort < 0 || c.SSHPort > 65535 {
return errors.New("ssh port: 1-65535 expected")
}
cfg := &config.Config{Kinds: model.Kinds, Reference: config.Cluster{Name: "\x00ref"},
Clients: []config.Client{{Cluster: config.Cluster{Name: c.Name}, Mapping: c.Mapping, Policies: c.Policies}}}
return cfg.Validate()
}
@@ -460,54 +493,105 @@ func (s *Server) handleDeleteCluster(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// handleCheckCluster tests the access, the version and the token privileges.
func (s *Server) handleCheckCluster(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
cfg, err := s.store.Config(r.Context())
if err != nil && !errors.Is(err, store.ErrNoReference) {
writeError(w, http.StatusInternalServerError, err.Error(), "config")
// handleRole makes a cluster the primary, a target or disabled.
func (s *Server) handleRole(w http.ResponseWriter, r *http.Request) {
var req struct {
Role string `json:"role"`
}
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
var cl config.Cluster
write := true
name := r.PathValue("name")
err := s.store.SetRole(r.Context(), name, req.Role)
s.audit(r, Identity{}, "cluster-role", name, err == nil, errText(err, req.Role))
switch {
case cfg != nil && cfg.Reference.Name == name:
cl, write = cfg.Reference, false
case cfg != nil:
c, ok := cfg.Client(name)
if !ok {
case errors.Is(err, store.ErrNotFound):
writeError(w, http.StatusNotFound, err.Error(), "notfound")
return
case err != nil:
writeError(w, http.StatusBadRequest, err.Error(), "invalid")
return
}
s.invalidate()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// handleCheckCluster tests the API (root@pam, PVE 9.x) and SSH. On the first
// connection the TLS certificate (mode "fp") and the SSH host key are recorded
// (trust on first use); trust=true accepts a new SSH host key.
func (s *Server) handleCheckCluster(w http.ResponseWriter, r *http.Request) {
var req struct {
Trust bool `json:"trust"`
}
if r.ContentLength > 0 {
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
}
name := r.PathValue("name")
c, cc, err := s.store.Cluster(r.Context(), name)
if errors.Is(err, store.ErrNotFound) {
writeError(w, http.StatusNotFound, "unknown cluster", "notfound")
return
}
cl = c.Cluster
default:
writeError(w, http.StatusBadRequest, "define a reference cluster first", "config")
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
ctx, cancel := context.WithTimeout(r.Context(), 45*time.Second)
defer cancel()
res := service.CheckCluster(ctx, cl, write, cfg.HasKind(model.KindDatacenter))
s.audit(r, Identity{}, "cluster-check", name, res.Error == "", res.Error)
res := service.CheckCluster(ctx, cc, c.TLS == store.TLSPin, req.Trust)
chk := &store.Check{At: time.Now().UTC(), OK: res.OK, Error: res.Error, Warnings: res.Warnings,
Version: res.Version, API: res.API, SSH: res.SSH}
for _, n := range res.Nodes {
chk.Nodes = append(chk.Nodes, n.Name)
}
// a TLS certificate is pinned only once the API answered with it
fp := ""
if res.API {
fp = res.TLSFingerprint
}
if err := s.store.Pin(r.Context(), name, fp, res.SSHHostKey, chk); err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
detail := res.Error
if fp != "" {
detail += " TLS certificate recorded " + fp
}
if res.SSHHostKey != "" {
detail += " SSH key recorded " + res.SSHFingerprint
}
s.audit(r, Identity{}, "cluster-check", name, res.OK, strings.TrimSpace(detail))
if fp != "" || res.SSHHostKey != "" {
s.invalidate()
}
writeJSON(w, http.StatusOK, res)
}
// handleForget clears the pinned certificate and SSH host key of a cluster
// (node reinstalled, new certificate): they are recorded again on the next test.
func (s *Server) handleForget(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
err := s.store.Forget(r.Context(), name)
s.audit(r, Identity{}, "cluster-forget", name, err == nil, errText(err, "TLS fingerprint and SSH key forgotten"))
if err != nil {
writeError(w, http.StatusNotFound, err.Error(), "notfound")
return
}
s.invalidate()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handleSettings(w http.ResponseWriter, r *http.Request) {
st, err := s.store.Settings(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
writeJSON(w, http.StatusOK, map[string]any{"settings": st, "allKinds": model.Kinds, "defaults": defaultPolicies()})
}
// defaultPolicies describes the built-in policies, shown in the settings.
func defaultPolicies() map[model.Kind]string {
return map[model.Kind]string{
model.KindDatacenter: "tag-style, registered-tags, user-tag-access, next-id : sync ; le reste : ignore ; webauthn, u2f, http_proxy : jamais",
model.KindZone: "nodes, bridge, peers, fabric, dns, reversedns, dnszone : override (mapping du client) ; le reste : sync",
model.KindVNet: "sync",
model.KindSubnet: "sync",
}
writeJSON(w, http.StatusOK, map[string]any{"settings": st, "allKinds": model.Kinds, "deps": adapter.KindDeps, "bundles": adapter.KindBundles})
}
func (s *Server) handleSaveSettings(w http.ResponseWriter, r *http.Request) {
@@ -516,17 +600,18 @@ func (s *Server) handleSaveSettings(w http.ResponseWriter, r *http.Request) {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
st.Kinds = adapter.Bundle(st.Kinds)
cfg := &config.Config{Kinds: st.Kinds, Objects: st.Objects, Policies: st.Policies, Prune: st.Prune,
Reference: config.Cluster{Name: "\x00ref", Secret: "x"}}
if len(cfg.Kinds) == 0 {
cfg.Kinds = model.Kinds
}
Reference: config.Cluster{Name: "\x00ref"}}
err := cfg.Validate()
if err == nil && len(st.Kinds) == 0 {
err = errors.New("select at least one kind")
}
if err == nil {
err = s.store.SaveSettings(r.Context(), st)
}
b, _ := json.Marshal(st)
s.audit(r, Identity{}, "settings-save", "synchronisation", err == nil, errText(err, string(b)))
s.audit(r, Identity{}, "settings-save", "replication", err == nil, errText(err, string(b)))
if err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "invalid")
return
@@ -535,62 +620,6 @@ func (s *Server) handleSaveSettings(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// handleExport returns the configuration as a CLI YAML file (no secret).
func (s *Server) handleExport(w http.ResponseWriter, r *http.Request) {
cfg, err := s.store.ExportConfig(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
b, err := yaml.Marshal(cfg)
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "yaml")
return
}
s.audit(r, Identity{}, "config-export", "", true, "")
w.Header().Set("Content-Type", "application/yaml; charset=utf-8")
w.Header().Set("Content-Disposition", `attachment; filename="confsync.yaml"`)
_, _ = fmt.Fprintf(w, "# pve-confsync : configuration exportée le %s par %s\n# Secrets des tokens non exportés : renseigner les variables token_secret_env (ou token_secret_file).\n\n",
time.Now().Format("2006-01-02 15:04"), identity(r).User)
_, _ = w.Write(b)
}
func (s *Server) handleImport(w http.ResponseWriter, r *http.Request) {
var req struct {
YAML string `json:"yaml"`
Replace bool `json:"replace"`
}
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
var cfg config.Config
dec := yaml.NewDecoder(strings.NewReader(req.YAML))
dec.KnownFields(true)
err := dec.Decode(&cfg)
if err == nil {
if len(cfg.Kinds) == 0 {
cfg.Kinds = append(cfg.Kinds, model.Kinds...)
}
err = cfg.Validate()
}
var warns []string
if err == nil {
warns, err = s.store.ImportConfig(r.Context(), &cfg, req.Replace)
}
s.audit(r, Identity{}, "config-import", fmt.Sprintf("%d cluster(s), remplacement=%t", len(cfg.Clients)+1, req.Replace),
err == nil, errText(err, strings.Join(warns, " ; ")))
if err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "invalid")
return
}
s.invalidate()
if warns == nil {
warns = []string{}
}
writeJSON(w, http.StatusOK, map[string]any{"status": "ok", "warnings": warns})
}
// ---- users and audit ----
func (s *Server) handleUsers(w http.ResponseWriter, r *http.Request) {
@@ -616,9 +645,9 @@ func (s *Server) handleSaveUser(w http.ResponseWriter, r *http.Request) {
}
name := r.PathValue("name")
err := s.store.SaveUser(r.Context(), name, req.Role, req.Password)
detail := "rôle " + req.Role
detail := "role " + req.Role
if req.Password != "" {
detail += ", mot de passe défini"
detail += ", password set"
}
s.audit(r, Identity{}, "user-save", name, err == nil, errText(err, detail))
if err != nil {
internal/server/auth.go
+52 -2
View File
@@ -183,7 +183,7 @@ func sessionKey(dir string) []byte {
// public paths: reachable without a session.
func public(p string) bool {
switch p {
case "/healthz", loginPage, "/style.css", "/boot.js", "/login.js", "/favicon.svg", "/auth/config", "/auth/login", "/auth/logout", "/auth/oidc":
case "/healthz", loginPage, "/style.css", "/boot.js", "/login.js", "/favicon.svg", "/auth/config", "/auth/login", "/auth/logout", "/auth/oidc", "/auth/setup":
return true
}
return false
@@ -197,6 +197,9 @@ func (a *Auth) ServeHTTP(w http.ResponseWriter, r *http.Request) {
case r.URL.Path == "/auth/logout":
a.handleLogout(w, r)
return
case r.URL.Path == "/auth/setup":
a.handleSetup(w, r)
return
case r.URL.Path == "/auth/oidc" && a.cfg.OIDCEnabled():
a.startLogin(w, r)
return
@@ -204,7 +207,8 @@ func (a *Auth) ServeHTTP(w http.ResponseWriter, r *http.Request) {
a.handleCallback(w, r)
return
case r.URL.Path == "/auth/config":
writeJSON(w, http.StatusOK, map[string]any{"local": a.cfg.Local, "oidc": a.cfg.OIDCEnabled(), "demo": a.cfg.Demo})
writeJSON(w, http.StatusOK, map[string]any{"local": a.cfg.Local, "oidc": a.cfg.OIDCEnabled(), "demo": a.cfg.Demo,
"setup": a.needSetup(r.Context())})
return
}
id, ok := a.sessionIdentity(r)
@@ -299,6 +303,52 @@ func (a *Auth) handleLogin(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, ret, http.StatusSeeOther)
}
// needSetup: local accounts enabled and none yet, the login page asks for the
// first administrator.
func (a *Auth) needSetup(ctx context.Context) bool {
if !a.cfg.Local {
return false
}
n, err := a.store.CountUsers(ctx)
return err == nil && n == 0
}
// handleSetup creates the first administrator on an empty database, then opens
// its session. Refused as soon as an account exists.
func (a *Auth) handleSetup(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost || !a.needSetup(r.Context()) {
http.Redirect(w, r, loginPage, http.StatusFound)
return
}
user, pass := strings.TrimSpace(r.PostFormValue("user")), r.PostFormValue("pass")
fail := func(code string) {
http.Redirect(w, r, loginPage+"?"+url.Values{"setup": {code}}.Encode(), http.StatusSeeOther)
}
if pass != r.PostFormValue("pass2") {
fail("mismatch")
return
}
if err := a.store.CreateFirstAdmin(r.Context(), user, pass); err != nil {
log.Printf("auth: first administrator: %v", err)
if errors.Is(err, store.ErrSetupDone) {
http.Redirect(w, r, loginPage, http.StatusSeeOther)
return
}
fail("invalid")
return
}
u, err := a.store.User(r.Context(), user)
if err != nil {
fail("invalid")
return
}
id := Identity{User: u.Name, Role: u.Role, Mode: "local"}
log.Printf("auth: first administrator %q created from %s", u.Name, clientIP(r))
a.audit(r, id, "setup", u.Name, true, "first administrator created")
a.setSession(w, r, session{User: u.Name, Role: u.Role, Mode: "local", Epoch: u.Epoch})
http.Redirect(w, r, "/#clusters", http.StatusSeeOther)
}
// handleLogout ends the session; with OIDC the provider's session ends on its logout page if it has one.
func (a *Auth) handleLogout(w http.ResponseWriter, r *http.Request) {
id, ok := a.sessionIdentity(r)
internal/server/nodes.go
+580
View File
@@ -0,0 +1,580 @@
package server
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"strings"
"sync"
"time"
"pve-confsync/internal/nodesync"
"pve-confsync/internal/pve"
"pve-confsync/internal/service"
"pve-confsync/internal/sshx"
"pve-confsync/internal/store"
)
// ---- node replication: nodes (SSH access), profiles, state, dry-run, apply ----
// handleNodes lists the nodes, the profiles and the catalog of items.
func (s *Server) handleNodes(w http.ResponseWriter, r *http.Request) {
nodes, err := s.store.Nodes(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
profiles, err := s.store.Profiles(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
if identity(r).Role != store.RoleAdmin {
for i, n := range nodes {
nodes[i] = store.Node{Key: n.Key, Cluster: n.Cluster, Node: n.Node, ClusterRole: n.ClusterRole, LastCheck: n.LastCheck}
}
}
writeJSON(w, http.StatusOK, map[string]any{"nodes": nodes, "profiles": profiles, "items": nodesync.Items,
"defaults": nodesync.DefaultPolicies})
}
// handleDiscover reads the nodes of every cluster (GET /cluster/status) and
// records them with their address in the cluster.
func (s *Server) handleDiscover(w http.ResponseWriter, r *http.Request) {
cls, err := s.store.Clusters(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
ctx, cancel := context.WithTimeout(r.Context(), readTimeout)
defer cancel()
errs := map[string]string{}
found := 0
for _, c := range cls {
if c.Role == store.ClusterDisabled {
continue
}
_, cc, err := s.store.Cluster(ctx, c.Name)
var nodes []service.DiscoveredNode
if err == nil {
nodes, err = service.DiscoverNodes(ctx, cc)
}
if err == nil {
var d []store.Discovered
for _, n := range nodes {
d = append(d, store.Discovered{Name: n.Name, IP: n.IP})
}
err = s.store.SetNodes(ctx, c.Name, d)
found += len(d)
}
if err != nil {
errs[c.Name] = err.Error()
}
}
s.audit(r, Identity{}, "nodes-discover", "all clusters", len(errs) == 0, fmt.Sprintf("%d node(s)", found))
s.invalidateNodes()
writeJSON(w, http.StatusOK, map[string]any{"found": found, "errors": errs})
}
func (s *Server) handleSaveNode(w http.ResponseWriter, r *http.Request) {
var req struct {
SSHHost string `json:"sshHost"`
SSHPort int `json:"sshPort"`
Password string `json:"password"`
ClearPassword bool `json:"clearPassword"`
}
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
cl, node := r.PathValue("cluster"), r.PathValue("node")
err := s.store.SaveNode(r.Context(), cl, node, req.SSHHost, req.SSHPort, req.Password, req.ClearPassword)
detail := fmt.Sprintf("ssh=%s:%d", req.SSHHost, req.SSHPort)
if req.Password != "" {
detail += " password=changed"
}
if req.ClearPassword {
detail += " password=cluster"
}
s.audit(r, Identity{}, "node-save", store.NodeKey(cl, node), err == nil, errText(err, detail))
switch {
case errors.Is(err, store.ErrNotFound):
writeError(w, http.StatusNotFound, "unknown node", "notfound")
case err != nil:
writeError(w, http.StatusBadRequest, err.Error(), "invalid")
default:
s.invalidateNodes()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
}
// handleCheckNode tests SSH on a node; the first test records its host key.
func (s *Server) handleCheckNode(w http.ResponseWriter, r *http.Request) {
var req struct {
Trust bool `json:"trust"`
}
if r.ContentLength > 0 {
if err := readJSON(r, &req); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
}
cl, node := r.PathValue("cluster"), r.PathValue("node")
a, err := s.store.NodeAccessOf(r.Context(), cl, node)
if errors.Is(err, store.ErrNotFound) {
writeError(w, http.StatusNotFound, "unknown node", "notfound")
return
}
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 30*time.Second)
defer cancel()
res := service.CheckNode(ctx, node, a.SSH, req.Trust)
chk := &store.Check{At: time.Now().UTC(), OK: res.OK, Error: res.Error, Version: res.Version, SSH: res.OK}
if err := s.store.PinNode(r.Context(), cl, node, res.SSHHostKey, chk); err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
detail := res.Error
if res.SSHHostKey != "" {
detail += " SSH key recorded " + res.SSHFingerprint
}
s.audit(r, Identity{}, "node-check", store.NodeKey(cl, node), res.OK, strings.TrimSpace(detail))
s.invalidateNodes()
writeJSON(w, http.StatusOK, res)
}
func (s *Server) handleForgetNode(w http.ResponseWriter, r *http.Request) {
cl, node := r.PathValue("cluster"), r.PathValue("node")
err := s.store.ForgetNode(r.Context(), cl, node)
s.audit(r, Identity{}, "node-forget", store.NodeKey(cl, node), err == nil, errText(err, "SSH key forgotten"))
if err != nil {
writeError(w, http.StatusNotFound, err.Error(), "notfound")
return
}
s.invalidateNodes()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handleSaveProfile(w http.ResponseWriter, r *http.Request) {
var p nodesync.Profile
if err := readJSON(r, &p); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
old := r.PathValue("name")
if old == "-" {
old = ""
}
err := s.store.SaveProfile(r.Context(), old, p)
b, _ := json.Marshal(p)
s.audit(r, Identity{}, "profile-save", p.Name, err == nil, errText(err, string(b)))
if err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "invalid")
return
}
s.invalidateNodes()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
func (s *Server) handleDeleteProfile(w http.ResponseWriter, r *http.Request) {
name := r.PathValue("name")
err := s.store.DeleteProfile(r.Context(), name)
s.audit(r, Identity{}, "profile-delete", name, err == nil, errText(err, ""))
if err != nil {
writeError(w, http.StatusNotFound, err.Error(), "notfound")
return
}
s.invalidateNodes()
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
}
// endpoints builds the access to every node: one API client per cluster,
// SSH as root on each node whose host key is recorded.
func (s *Server) endpoints(ctx context.Context) (map[string]nodesync.Endpoint, error) {
acc, err := s.store.NodeAccesses(ctx)
if err != nil {
return nil, err
}
apis := map[string]*pve.Client{}
out := map[string]nodesync.Endpoint{}
for key, a := range acc {
c, ok := apis[a.Cluster.Name]
if !ok {
if c, err = a.Cluster.PVE(); err != nil {
return nil, err
}
apis[a.Cluster.Name] = c
}
ep := nodesync.Endpoint{Key: key, Cluster: a.Node.Cluster, Node: a.Node.Node, API: c}
if a.SSH.HostKey != "" {
if sh, err := sshx.New(a.SSH); err == nil {
ep.Shell = sh
}
}
out[key] = ep
}
return out, nil
}
func (s *Server) invalidateNodes() {
s.nodeMu.Lock()
s.nodeStates = nil
s.nodeMu.Unlock()
}
// handleNodeState returns the last comparison of a profile; refresh=1 reads
// the nodes again.
func (s *Server) handleNodeState(w http.ResponseWriter, r *http.Request) {
name := r.URL.Query().Get("profile")
s.nodeMu.Lock()
st := s.nodeStates[name]
s.nodeMu.Unlock()
if st == nil || r.URL.Query().Get("refresh") == "1" {
prof, err := s.store.Profile(r.Context(), name)
if errors.Is(err, store.ErrNotFound) {
writeError(w, http.StatusNotFound, "unknown profile", "notfound")
return
}
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
eps, err := s.endpoints(r.Context())
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "config")
return
}
ctx, cancel := context.WithTimeout(r.Context(), readTimeout)
defer cancel()
st = nodesync.Analyze(ctx, &prof, eps, nil)
s.rememberNodes(st, true)
}
writeJSON(w, http.StatusOK, map[string]any{"state": st})
}
// rememberNodes keeps a comparison; a partial one (some targets) only
// replaces the targets it read.
func (s *Server) rememberNodes(st *nodesync.State, all bool) {
s.nodeMu.Lock()
defer s.nodeMu.Unlock()
if s.nodeStates == nil {
s.nodeStates = map[string]*nodesync.State{}
}
old := s.nodeStates[st.Profile]
if all || old == nil {
if all {
s.nodeStates[st.Profile] = st
}
return
}
cp := *old
cp.Master = st.Master
cp.Targets = append([]nodesync.Target(nil), old.Targets...)
for _, t := range st.Targets {
for i := range cp.Targets {
if cp.Targets[i].Key == t.Key {
cp.Targets[i] = t
}
}
}
s.nodeStates[st.Profile] = &cp
}
type nodeSelection struct {
Profile string `json:"profile"`
Scope string `json:"scope"` // global (several targets), client (one target) or object
Targets []string `json:"targets"` // global: the targets (empty: all of them)
Client string `json:"client"` // client and object scopes
Objects []string `json:"objects"` // object scope: "item/id"
Digests map[string]string `json:"digests"`
Confirm string `json:"confirm"` // network changes: target (profile name for several)
}
func (sel *nodeSelection) check() error {
if sel.Profile == "" {
return errors.New("profile required")
}
s := selection{Scope: sel.Scope, Targets: sel.Targets, Client: sel.Client, Objects: sel.Objects}
if err := s.check(); err != nil {
return err
}
sel.Targets, sel.Client, sel.Objects = s.Targets, s.Client, s.Objects
return nil
}
func (sel nodeSelection) clients() []string {
if sel.Client == "" {
return sel.Targets
}
return []string{sel.Client}
}
func (sel nodeSelection) target() string {
t := sel.Profile + ": "
switch {
case sel.Scope == "client":
t += sel.Client
case sel.Scope == "object":
t += sel.Client + ": " + strings.Join(sel.Objects, ", ")
case len(sel.Targets) > 0:
t += "targets " + strings.Join(sel.Targets, ", ")
default:
t += "all targets"
}
return t
}
func (s *Server) nodePlan(ctx context.Context, sel nodeSelection) (*nodesync.State, []nodesync.TargetPlan, error) {
prof, err := s.store.Profile(ctx, sel.Profile)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
err = fmt.Errorf("unknown profile %q", sel.Profile)
}
return nil, nil, err
}
for _, c := range sel.clients() {
found := false
for _, t := range prof.Targets {
found = found || t == c
}
if !found {
return nil, nil, fmt.Errorf("unknown target %q", c)
}
}
eps, err := s.endpoints(ctx)
if err != nil {
return nil, nil, err
}
rctx, cancel := context.WithTimeout(ctx, readTimeout)
defer cancel()
st := nodesync.Analyze(rctx, &prof, eps, sel.clients())
s.rememberNodes(st, sel.Scope == "global" && len(sel.Targets) == 0)
return st, st.Plans(sel.Objects), nil
}
// node runs are journaled with the scopes "nodes", "node" and "node-object".
func nodeScope(scope string) string {
switch scope {
case "global":
return "nodes"
case "client":
return "node"
}
return "node-object"
}
type nodeRunDetail struct {
Profile string `json:"profile"`
Plans []nodesync.TargetPlan `json:"plans"`
Results map[string]*nodeResult `json:"results,omitempty"`
}
type nodeResult struct {
Status string `json:"status"`
Error string `json:"error,omitempty"`
Steps []nodesync.Step `json:"steps"`
}
func nodeSummary(plans []nodesync.TargetPlan) (string, string) {
changes, targets, blocked := 0, 0, 0
for _, p := range plans {
if p.Error != "" || len(p.Plan.Errors) > 0 {
blocked++
continue
}
if n := len(p.Plan.Changes); n > 0 {
changes += n
targets++
}
}
switch {
case blocked > 0:
return store.RunBlocked, fmt.Sprintf("%d change(s) on %d target(s), %d target(s) in error", changes, targets, blocked)
case changes == 0:
return store.RunNoop, "no difference"
}
return store.RunOK, fmt.Sprintf("%d change(s) on %d target(s)", changes, targets)
}
func nodePlanTargets(plans []nodesync.TargetPlan) []string {
out := make([]string, 0, len(plans))
for _, p := range plans {
out = append(out, p.Client)
}
return out
}
func (s *Server) handleNodeDryRun(w http.ResponseWriter, r *http.Request) {
var sel nodeSelection
if err := readJSON(r, &sel); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
if err := sel.check(); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
id := identity(r)
_, plans, err := s.nodePlan(r.Context(), sel)
if err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "config")
return
}
status, summary := nodeSummary(plans)
d, _ := json.Marshal(nodeRunDetail{Profile: sel.Profile, Plans: plans})
runID, err := s.store.StartRun(r.Context(), store.Run{User: id.User, Mode: "dry-run", Scope: nodeScope(sel.Scope),
Clients: nodePlanTargets(plans), Objects: sel.Objects, Status: status, Summary: summary, Detail: d})
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
_ = s.store.UpdateRun(r.Context(), runID, status, summary, d)
s.audit(r, id, "node-dry-run", sel.target(), true, summary)
writeJSON(w, http.StatusOK, map[string]any{"run": runID, "plans": plans, "summary": summary})
}
// handleNodeApply recomputes the plans, checks the digests of the dry-run and
// the typed confirmation of the network changes, then applies them in the
// background, one target after the other.
func (s *Server) handleNodeApply(w http.ResponseWriter, r *http.Request) {
var sel nodeSelection
if err := readJSON(r, &sel); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
if err := sel.check(); err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "request")
return
}
id := identity(r)
if !s.applyMu.TryLock() {
writeError(w, http.StatusConflict, "a replication is already running", "busy")
return
}
started := false
defer func() {
if !started {
s.applyMu.Unlock()
}
}()
st, plans, err := s.nodePlan(r.Context(), sel)
if err != nil {
writeError(w, http.StatusBadRequest, err.Error(), "config")
return
}
var todo []nodesync.TargetPlan
network := false
for _, p := range plans {
switch {
case p.Error != "":
s.audit(r, id, "node-apply", sel.target(), false, p.Client+": "+p.Error)
writeError(w, http.StatusConflict, p.Client+": "+p.Error, "read")
return
case len(p.Plan.Errors) > 0:
s.audit(r, id, "node-apply", sel.target(), false, p.Client+": plan en erreur")
writeError(w, http.StatusConflict, p.Client+": "+p.Plan.Errors[0].Message, "plan")
return
case sel.Digests[p.Client] != p.Digest:
s.audit(r, id, "node-apply", sel.target(), false, p.Client+": the state changed since the dry-run")
writeJSON(w, http.StatusConflict, map[string]any{"error": p.Client + ": the state changed since the dry-run, run it again",
"code": "stale", "plans": plans})
return
case p.Plan.Empty():
continue
}
network = network || p.Network
todo = append(todo, p)
}
if len(todo) == 0 {
writeJSON(w, http.StatusOK, map[string]any{"run": 0, "summary": "no difference"})
return
}
if network {
want := sel.Client
if sel.Scope == "global" {
want = sel.Profile
}
if sel.Confirm != want {
writeError(w, http.StatusBadRequest, "network changes: confirmation required ("+want+")", "confirm")
return
}
}
det := &nodeRunDetail{Profile: sel.Profile, Plans: todo, Results: map[string]*nodeResult{}}
for _, p := range todo {
det.Results[p.Client] = &nodeResult{Status: "pending", Steps: []nodesync.Step{}}
}
d, _ := json.Marshal(det)
runID, err := s.store.StartRun(r.Context(), store.Run{User: id.User, Mode: "apply", Scope: nodeScope(sel.Scope),
Clients: nodePlanTargets(todo), Objects: sel.Objects, Status: store.RunRunning, Summary: "en cours", Detail: d})
if err != nil {
writeError(w, http.StatusInternalServerError, err.Error(), "store")
return
}
s.audit(r, id, "node-apply-start", sel.target(), true, fmt.Sprintf("journal #%d", runID))
started = true
s.jobs.Add(1)
rr := r.Clone(context.WithoutCancel(r.Context()))
go func() {
defer s.jobs.Done()
defer s.applyMu.Unlock()
s.runNodeApply(rr, id, runID, sel, st, det)
}()
writeJSON(w, http.StatusAccepted, map[string]any{"run": runID})
}
// runNodeApply applies the plans one target after the other and stops at the
// first failure: the nodes are often of the same cluster, a configuration
// that breaks one would break the next.
func (s *Server) runNodeApply(r *http.Request, id Identity, runID int64, sel nodeSelection, st *nodesync.State, det *nodeRunDetail) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
defer cancel()
var mu sync.Mutex
save := func(status, summary string) {
mu.Lock()
d, _ := json.Marshal(det)
mu.Unlock()
if err := s.store.UpdateRun(ctx, runID, status, summary, d); err != nil {
log.Printf("journal #%d: %v", runID, err)
}
}
failed, done := 0, 0
for _, p := range det.Plans {
res := det.Results[p.Client]
if failed > 0 {
res.Status = "skipped"
continue
}
res.Status = store.RunRunning
save(store.RunRunning, p.Client+" en cours")
err := st.Apply(ctx, p, sel.Objects, func(step nodesync.Step) {
mu.Lock()
res.Steps = append(res.Steps, step)
mu.Unlock()
save(store.RunRunning, p.Client+" : "+step.Action)
})
if err != nil {
failed++
res.Status, res.Error = store.RunError, err.Error()
log.Printf("node apply %s (journal #%d): %v", p.Client, runID, err)
s.audit(r, id, "node-apply", p.Client, false, err.Error())
continue
}
done++
res.Status = store.RunOK
s.audit(r, id, "node-apply", p.Client, true, fmt.Sprintf("%d change(s), verified", len(p.Plan.Changes)))
}
status := store.RunOK
summary := fmt.Sprintf("%d target(s) replicated and verified", done)
if failed > 0 {
status = store.RunError
summary = fmt.Sprintf("%d target(s) failed, %d replicated", failed, done)
}
save(status, summary)
s.invalidateNodes()
}
internal/server/server.go
+24 -6
View File
@@ -1,7 +1,8 @@
// Package server is the web UI of pve-confsync (phases 1 and 2): cluster list,
// visual diff per client, dry-run and synchronization per object, per client or
// global, execution journal and audit trail. The pages (web/) are embedded in
// the binary; the API is JSON under /api/.
// Package server is the web UI of pve-confsync, the only way to use it: first
// administrator, clusters (root credentials, primary and targets), replicated
// kinds, visual diff per target, dry-run and synchronization per object, per
// target or towards several targets, execution journal and audit trail. The
// pages (web/) are embedded in the binary; the API is JSON under /api/.
package server
import (
@@ -15,6 +16,7 @@ import (
"sync"
"time"
"pve-confsync/internal/nodesync"
"pve-confsync/internal/service"
"pve-confsync/internal/store"
)
@@ -39,6 +41,10 @@ type Server struct {
state *service.State
stateAt time.Time
// last comparison of each node replication profile
nodeMu sync.Mutex
nodeStates map[string]*nodesync.State
applyMu sync.Mutex // one synchronization at a time
jobs sync.WaitGroup
}
@@ -77,10 +83,22 @@ func (s *Server) Handler() http.Handler {
mux.Handle("PUT /api/clusters/{name}", s.role(admin, s.handleSaveCluster))
mux.Handle("DELETE /api/clusters/{name}", s.role(admin, s.handleDeleteCluster))
mux.Handle("POST /api/clusters/{name}/check", s.role(admin, s.handleCheckCluster))
mux.Handle("POST /api/clusters/{name}/role", s.role(admin, s.handleRole))
mux.Handle("POST /api/clusters/{name}/forget", s.role(admin, s.handleForget))
mux.Handle("GET /api/settings", s.role(viewer, s.handleSettings))
mux.Handle("PUT /api/settings", s.role(admin, s.handleSaveSettings))
mux.Handle("GET /api/config/export", s.role(admin, s.handleExport))
mux.Handle("POST /api/config/import", s.role(admin, s.handleImport))
mux.Handle("GET /api/nodes", s.role(viewer, s.handleNodes))
mux.Handle("POST /api/nodes/discover", s.role(admin, s.handleDiscover))
mux.Handle("PUT /api/nodes/{cluster}/{node}", s.role(admin, s.handleSaveNode))
mux.Handle("POST /api/nodes/{cluster}/{node}/check", s.role(admin, s.handleCheckNode))
mux.Handle("POST /api/nodes/{cluster}/{node}/forget", s.role(admin, s.handleForgetNode))
mux.Handle("PUT /api/profiles/{name}", s.role(admin, s.handleSaveProfile))
mux.Handle("DELETE /api/profiles/{name}", s.role(admin, s.handleDeleteProfile))
mux.Handle("GET /api/nodestate", s.role(viewer, s.handleNodeState))
mux.Handle("POST /api/nodedryrun", s.role(operator, s.handleNodeDryRun))
// writing system files as root on the nodes: administrators only
mux.Handle("POST /api/nodeapply", s.role(admin, s.handleNodeApply))
mux.Handle("GET /api/users", s.role(admin, s.handleUsers))
mux.Handle("PUT /api/users/{name}", s.role(admin, s.handleSaveUser))
internal/server/server_test.go
+128 -2
View File
@@ -4,17 +4,20 @@ import (
"bytes"
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/cookiejar"
"net/http/httptest"
"net/url"
"path/filepath"
"strings"
"testing"
"testing/fstest"
"time"
"pve-confsync/internal/demo"
"pve-confsync/internal/pve/pvetest"
"pve-confsync/internal/store"
)
@@ -117,8 +120,8 @@ func TestAuthAndRoles(t *testing.T) {
t.Fatalf("clusters: %d", code)
}
for _, c := range d["clusters"].([]any) {
if s, ok := c.(map[string]any)["secret"]; ok && s != "" {
t.Fatalf("secret returned: %v", c)
if s, ok := c.(map[string]any)["password"]; ok && s != "" {
t.Fatalf("password returned: %v", c)
}
}
}
@@ -147,6 +150,9 @@ func TestDryRunAndApply(t *testing.T) {
if code != http.StatusOK {
t.Fatalf("dry-run site-a: %d %v", code, d)
}
if b, _ := json.Marshal(d); strings.Contains(string(b), "HashOnlyForTheDemo") || !strings.Contains(string(b), "pve-password") {
t.Fatalf("password hashes must be planned but never shown: %s", b)
}
sel["digests"] = digests(d)
if code, d := op.do("POST", "/api/apply", sel, true); code != http.StatusBadRequest || d["code"] != "confirm" {
t.Fatalf("SDN apply without confirmation: %d %v", code, d)
@@ -188,4 +194,124 @@ func TestDryRunAndApply(t *testing.T) {
}
}
// TestFromScratch: empty database, first administrator from the web, clusters
// added with their root password, certificate and SSH host key pinned by the
// first test, primary chosen, replication of the accounts to one target.
func TestFromScratch(t *testing.T) {
ctx := context.Background()
dir := t.TempDir()
st, err := store.Open(filepath.Join(dir, "t.db"), bytes.Repeat([]byte{7}, 32))
if err != nil {
t.Fatal(err)
}
srv, err := New(Options{Store: st, Version: "test",
Static: fstest.MapFS{"index.html": {Data: []byte("<html>")}, "login.html": {Data: []byte("<html>")}},
Auth: AuthConfig{Local: true, SessionTTL: time.Hour, DataDir: dir}})
if err != nil {
t.Fatal(err)
}
hs := httptest.NewServer(srv.Handler())
ref, tgt := pvetest.New("9.1.4"), pvetest.New("9.0.10")
t.Cleanup(func() { hs.Close(); srv.Wait(); ref.Close(); tgt.Close(); st.Close() })
ref.Groups["ops"] = map[string]any{"comment": "Ops"}
ref.Users["alice@pve"] = map[string]any{"enable": 1, "groups": "ops"}
ref.Shadow["alice"] = "$5$s$aliceSecretHash"
anon := &client{t: t, url: hs.URL, c: http.DefaultClient}
if _, d := anon.do("GET", "/auth/config", nil, false); d["setup"] != true {
t.Fatalf("setup expected: %v", d)
}
jar, _ := cookiejar.New(nil)
adm := &client{t: t, url: hs.URL, c: &http.Client{Jar: jar, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}}
r, err := adm.c.PostForm(hs.URL+"/auth/setup", url.Values{"user": {"cedric"}, "pass": {"a-long-password"}, "pass2": {"a-long-password"}})
if err != nil {
t.Fatal(err)
}
r.Body.Close()
if r.StatusCode != http.StatusSeeOther || r.Header.Get("Location") != "/#clusters" {
t.Fatalf("setup: %d %s", r.StatusCode, r.Header.Get("Location"))
}
if r, _ := anon.c.PostForm(hs.URL+"/auth/setup", url.Values{"user": {"intrus"}, "pass": {"a-long-password"}, "pass2": {"a-long-password"}}); r != nil {
r.Body.Close()
}
if n, _ := st.CountUsers(ctx); n != 1 {
t.Fatal("a second setup must be refused")
}
if _, d := adm.do("GET", "/api/state", nil, false); d["setup"] != "clusters" {
t.Fatalf("state on an empty database: %v", d)
}
// a primary alone: no target yet, the state lists none (not null); no port: 8006
if code, d := adm.do("PUT", "/api/clusters/-", map[string]any{"name": "solo", "role": "primary", "url": "https://pve-solo", "password": "x", "tls": "insecure"}, true); code != http.StatusOK {
t.Fatalf("save solo: %d %v", code, d)
}
if _, d := adm.do("GET", "/api/clusters", nil, false); d["clusters"].([]any)[0].(map[string]any)["url"] != "https://pve-solo:8006" {
t.Fatalf("default port expected: %v", d)
}
if _, d := adm.do("GET", "/api/state", nil, false); d["state"] == nil || d["state"].(map[string]any)["clients"] == nil {
t.Fatalf("clients must be an empty list: %v", d)
}
if code, d := adm.do("DELETE", "/api/clusters/solo", nil, true); code != http.StatusOK {
t.Fatalf("delete solo: %d %v", code, d)
}
for _, c := range []struct {
name string
s *pvetest.Server
}{{"ref", ref}, {"tgt", tgt}} {
_, port := c.s.SSHAddr()
code, d := adm.do("PUT", "/api/clusters/-", map[string]any{"name": c.name, "url": c.s.URL, "password": pvetest.Password,
"tls": "fp", "sshPort": port}, true)
if code != http.StatusOK {
t.Fatalf("save %s: %d %v", c.name, code, d)
}
code, d = adm.do("POST", "/api/clusters/"+c.name+"/check", nil, true)
if code != http.StatusOK || d["ok"] != true || d["tlsFingerprint"] == "" || d["sshFingerprint"] == "" {
t.Fatalf("check %s: %d %v", c.name, code, d)
}
if code, d = adm.do("POST", "/api/clusters/"+c.name+"/check", nil, true); d["ok"] != true || d["sshFingerprint"] != nil {
t.Fatalf("second check %s (pinned): %d %v", c.name, code, d)
}
}
if _, d := adm.do("GET", "/api/state", nil, false); d["setup"] != "primary" {
t.Fatalf("state without primary: %v", d)
}
if code, d := adm.do("POST", "/api/clusters/ref/role", map[string]any{"role": "primary"}, true); code != http.StatusOK {
t.Fatalf("role: %d %v", code, d)
}
// accounts left out of the run ("do not replicate"): their passwords go with them
skipped := map[string]any{"scope": "global", "targets": []string{"tgt"}, "skip": []string{"pve-user", "pve-group", "pve-role"}}
if code, d := adm.do("POST", "/api/dryrun", skipped, true); code != http.StatusOK || strings.Contains(fmt.Sprint(d["plans"]), "pve-") {
t.Fatalf("dry-run without the accounts: %d %v", code, d)
}
sel := map[string]any{"scope": "global", "targets": []string{"tgt"}}
code, d := adm.do("POST", "/api/dryrun", sel, true)
if code != http.StatusOK {
t.Fatalf("dry-run: %d %v", code, d)
}
sel["digests"] = digests(d)
if code, d = adm.do("POST", "/api/apply", sel, true); code != http.StatusAccepted {
t.Fatalf("apply: %d %v", code, d)
}
id := int(d["run"].(float64))
var run map[string]any
for i := 0; i < 100; i++ {
_, run = adm.do("GET", "/api/runs/"+itoa(id), nil, false)
if run["status"] != "running" {
break
}
time.Sleep(50 * time.Millisecond)
}
if run["status"] != "ok" {
t.Fatalf("run: %v", run)
}
if tgt.Shadow["alice"] != "$5$s$aliceSecretHash" || tgt.Users["alice@pve"]["groups"] != "ops" {
t.Fatalf("target: %v %v", tgt.Shadow, tgt.Users)
}
if b, _ := json.Marshal(run); strings.Contains(string(b), "aliceSecretHash") {
t.Fatal("hash in the journal")
}
if ref.Calls != nil {
t.Fatalf("the primary must never be written: %v", ref.Calls)
}
}
func itoa(i int) string { b, _ := json.Marshal(i); return string(b) }
internal/service/check_test.go
+39
View File
@@ -0,0 +1,39 @@
package service
import (
"context"
"crypto/ed25519"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"strings"
"testing"
"golang.org/x/crypto/ssh"
"pve-confsync/internal/config"
"pve-confsync/internal/pve/pvetest"
)
// A node that presents another SSH host key than the recorded one is refused,
// until the new key is explicitly trusted.
func TestCheckSSHHostKeyMismatch(t *testing.T) {
s := pvetest.New("9.1.4")
defer s.Close()
pub, _, _ := ed25519.GenerateKey(rand.Reader)
other, _ := ssh.NewPublicKey(pub)
sum := sha256.Sum256(s.Certificate().Raw)
host, port := s.SSHAddr()
cl := config.Cluster{Name: "c", URL: s.URL, User: "root@pam", Password: pvetest.Password,
Fingerprint: hex.EncodeToString(sum[:]), SSHHost: host, SSHPort: port,
SSHHostKey: strings.TrimSpace(string(ssh.MarshalAuthorizedKey(other)))}
res := CheckCluster(context.Background(), cl, false, false)
if res.OK || !res.SSHMismatch || !res.API || res.SSHHostKey != "" || !strings.Contains(res.Error, "host key") {
t.Fatalf("mismatch expected: %+v", res)
}
res = CheckCluster(context.Background(), cl, false, true)
if !res.OK || res.SSHHostKey != s.SSHHostKey() {
t.Fatalf("trusted key expected: %+v", res)
}
}
internal/service/nodes.go
+103
View File
@@ -0,0 +1,103 @@
package service
import (
"context"
"fmt"
"net/url"
"strings"
"pve-confsync/internal/config"
"pve-confsync/internal/pve"
"pve-confsync/internal/sshx"
)
// DiscoveredNode is a node listed by GET /cluster/status.
type DiscoveredNode struct {
Name string `json:"name"`
IP string `json:"ip"`
Online bool `json:"online"`
}
// DiscoverNodes lists the nodes of a cluster with their address in the
// cluster. A standalone node without address gets the host of the API URL.
func DiscoverNodes(ctx context.Context, cl config.Cluster) ([]DiscoveredNode, error) {
cl.SSHHostKey = ""
c, err := cl.PVE()
if err != nil {
return nil, err
}
var status []map[string]any
if err := c.Get(ctx, "cluster/status", nil, &status); err != nil {
return nil, err
}
var out []DiscoveredNode
for _, s := range status {
if pve.Str(s["type"]) != "node" {
continue
}
n := DiscoveredNode{Name: pve.Str(s["name"]), IP: pve.Str(s["ip"]), Online: pve.Str(s["online"]) == "1"}
if n.IP == "" {
if u, err := url.Parse(cl.URL); err == nil {
n.IP = u.Hostname()
}
}
out = append(out, n)
}
return out, nil
}
// NodeCheck is the result of the SSH test of a node.
type NodeCheck struct {
OK bool `json:"ok"`
Error string `json:"error,omitempty"`
Version string `json:"version,omitempty"`
SSHHostKey string `json:"-"`
SSHFingerprint string `json:"sshFingerprint,omitempty"`
SSHMismatch bool `json:"sshMismatch,omitempty"`
}
// CheckNode tests SSH as root on a node: the host key (recorded on the first
// connection, trust accepts a new one), pveversion, and the host name, which
// must be the name of the node in its cluster (a wrong address is refused).
func CheckNode(ctx context.Context, node string, o sshx.Options, trust bool) NodeCheck {
var res NodeCheck
fail := func(err error) NodeCheck {
res.Error = err.Error()
return res
}
if o.HostKey == "" || trust {
seen, fp, err := sshx.Scan(ctx, o)
if err != nil {
return fail(fmt.Errorf("SSH: %w", err))
}
o.HostKey, res.SSHHostKey, res.SSHFingerprint = seen, seen, fp
}
sh, err := sshx.New(o)
if err != nil {
return fail(fmt.Errorf("SSH: %w", err))
}
out, err := sh.Run(ctx, "pveversion; hostname", nil)
if err != nil {
if strings.Contains(err.Error(), "host key mismatch") {
_, fp, _ := sshx.Scan(ctx, o)
res.SSHMismatch, res.SSHFingerprint = true, fp
return fail(fmt.Errorf("SSH: node %s presents host key %s instead of the recorded %s", o.Host, fp, sshx.Fingerprint(o.HostKey)))
}
return fail(fmt.Errorf("SSH: %w", err))
}
lines := strings.Split(strings.TrimSpace(string(out)), "\n")
if !strings.HasPrefix(lines[0], "pve-manager/") {
return fail(fmt.Errorf("SSH: %s is not a PVE node (pveversion: %q)", o.Host, lines[0]))
}
res.Version = strings.SplitN(strings.TrimPrefix(lines[0], "pve-manager/"), "/", 2)[0]
host := ""
if len(lines) > 1 {
host = strings.TrimSpace(lines[len(lines)-1])
}
if short, _, _ := strings.Cut(host, "."); !strings.EqualFold(short, node) {
res.SSHHostKey = "" // not the expected node: nothing is recorded
return fail(fmt.Errorf("SSH: %s answers as %q, not as node %s", o.Host, host, node))
}
res.OK = true
return res
}
internal/service/service.go
+71 -45
View File
@@ -7,6 +7,7 @@ import (
"context"
"fmt"
"sort"
"strings"
"sync"
"time"
@@ -15,6 +16,7 @@ import (
"pve-confsync/internal/engine"
"pve-confsync/internal/model"
"pve-confsync/internal/pve"
"pve-confsync/internal/sshx"
)
// Node is the PVE version of one node.
@@ -95,7 +97,7 @@ type State struct {
// clients (empty: all). A cluster that cannot be read is reported in its
// Error field, it does not stop the others.
func Analyze(ctx context.Context, cfg *config.Config, only []string) *State {
st := &State{At: time.Now().UTC(), Kinds: cfg.Kinds, Prune: cfg.Prune, cfg: cfg, actuals: map[string][]model.Object{}}
st := &State{At: time.Now().UTC(), Kinds: cfg.Kinds, Prune: cfg.Prune, Clients: []Client{}, cfg: cfg, actuals: map[string][]model.Object{}}
var refSnap model.Snapshot
st.Reference, refSnap = read(ctx, cfg.Reference, cfg.Kinds, adapter.ReadRunning)
st.ref = refSnap.Objects
@@ -255,6 +257,10 @@ func rows(cfg *config.Config, cl *config.Client, ref, actual []model.Object, pla
case fd.Policy != adapter.Ignore && fd.Current != nil:
fd.Target = fd.Current
}
if adapter.IsSensitive(k, f) { // password hashes: a digest only
fd.Ref, fd.Mapping = adapter.Redact(fd.Ref), adapter.Redact(fd.Mapping)
fd.Current, fd.Target = adapter.Redact(fd.Current), adapter.Redact(fd.Target)
}
row.Fields = append(row.Fields, fd)
}
out = append(out, row)
@@ -309,7 +315,7 @@ type ClientPlan struct {
// Plans returns the plans of the given clients restricted to the given objects
// (empty: everything), as a dry-run shows them and an apply checks them.
func (st *State) Plans(keys []string) []ClientPlan {
func (st *State) Plans(keys []string, skip []model.Kind) []ClientPlan {
var out []ClientPlan
for _, c := range st.Clients {
cp := ClientPlan{Client: c.Name}
@@ -319,7 +325,7 @@ func (st *State) Plans(keys []string) []ClientPlan {
case c.Error != "":
cp.Error = c.Error
default:
cp.Plan = engine.Filter(c.Plan, keys)
cp.Plan = engine.Skip(c.Plan, keys, skip)
cp.Digest = cp.Plan.Digest()
cp.SDN = cp.Plan.HasSDN()
}
@@ -330,7 +336,7 @@ func (st *State) Plans(keys []string) []ClientPlan {
// Apply writes one plan on a client, then checks that the selected objects are
// in line with the reference (same reference objects as the plan).
func (st *State) Apply(ctx context.Context, cp ClientPlan, keys []string, log func(engine.Step)) error {
func (st *State) Apply(ctx context.Context, cp ClientPlan, keys []string, skip []model.Kind, log func(engine.Step)) error {
cl, ok := st.cfg.Client(cp.Client)
if !ok {
return fmt.Errorf("unknown client %q", cp.Client)
@@ -345,7 +351,7 @@ func (st *State) Apply(ctx context.Context, cp ClientPlan, keys []string, log fu
}
start := time.Now()
err = engine.Verify(ctx, c, func(actual []model.Object) engine.Plan {
return engine.Filter(engine.MakePlan(st.cfg, cl, st.ref, actual), keys)
return engine.Skip(engine.MakePlan(st.cfg, cl, st.ref, actual), keys, skip)
}, st.cfg.Kinds)
s := engine.Step{At: start.UTC(), Action: "verify"}
if err != nil {
@@ -355,67 +361,87 @@ func (st *State) Apply(ctx context.Context, cp ClientPlan, keys []string, log fu
return err
}
// Check tests the access to a cluster: versions (9.x only) and token privileges.
// Check is the result of a connection test: API with the root credentials
// (login, PVE 9.x on every node) and SSH as root on the node.
type Check struct {
Nodes []Node `json:"nodes,omitempty"`
OK bool `json:"ok"`
Error string `json:"error,omitempty"`
Warnings []string `json:"warnings,omitempty"`
API bool `json:"api"`
SSH bool `json:"ssh"`
Version string `json:"version,omitempty"`
Nodes []Node `json:"nodes,omitempty"`
// set when something is seen for the first time and must be recorded
TLSFingerprint string `json:"tlsFingerprint,omitempty"`
SSHHostKey string `json:"-"`
SSHFingerprint string `json:"sshFingerprint,omitempty"`
SSHMismatch bool `json:"sshMismatch,omitempty"` // the node presents another key than the recorded one
}
// CheckCluster runs the same checks as the "check" command.
func CheckCluster(ctx context.Context, cl config.Cluster, write, dcOptions bool) Check {
// CheckCluster tests a cluster. pinTLS: the TLS mode pins the certificate and
// none is recorded yet (trust on first use). trust accepts a new SSH host key.
func CheckCluster(ctx context.Context, cl config.Cluster, pinTLS, trust bool) Check {
var res Check
c, err := cl.PVE()
if err != nil {
fail := func(err error) Check {
res.Error = err.Error()
return res
}
if pinTLS && cl.Fingerprint == "" {
fp, err := pve.PeerFingerprint(ctx, cl.URL)
if err != nil {
return fail(fmt.Errorf("TLS: %w", err))
}
cl.Fingerprint, res.TLSFingerprint = fp, fp
}
if cl.Insecure {
res.Warnings = append(res.Warnings, "TLS certificate not verified (lab only)")
}
key := cl.SSHHostKey
cl.SSHHostKey = "" // API only for now
c, err := cl.PVE()
if err != nil {
return fail(err)
}
if err := c.Login(ctx); err != nil {
return fail(fmt.Errorf("API: %w", err))
}
vs, err := c.CheckVersion(ctx)
res.Nodes = nodes(vs)
if err != nil {
res.Error = err.Error()
return res
if len(vs) > 0 {
res.Version = vs[0].Version
}
perms, err := c.Permissions(ctx)
if err != nil {
res.Warnings = append(res.Warnings, fmt.Sprintf("token privileges unreadable (%v)", err))
return res
return fail(err)
}
res.Warnings = PrivilegeWarnings(perms, write, dcOptions)
return res
}
res.API = true
// PrivilegeWarnings flags a reference token able to write, and a client token
// missing the privileges needed by the synchronized kinds.
func PrivilegeWarnings(perms map[string]map[string]any, write, dcOptions bool) []string {
has := func(path, priv string) bool { _, ok := perms[path][priv]; return ok }
anyPath := func(priv string) bool {
for p := range perms {
if has(p, priv) {
return true
o := cl.SSHOptions()
if key == "" || trust {
seen, fp, err := sshx.Scan(ctx, o)
if err != nil {
return fail(fmt.Errorf("SSH: %w", err))
}
key, res.SSHHostKey, res.SSHFingerprint = seen, seen, fp
}
return false
o.HostKey = key
sh, err := sshx.New(o)
if err != nil {
return fail(fmt.Errorf("SSH: %w", err))
}
var w []string
if !write {
for _, p := range []string{"Sys.Modify", "SDN.Allocate", "Permissions.Modify"} {
if anyPath(p) {
w = append(w, "le token de référence possède "+p+" : un token en lecture seule est attendu")
out, err := sh.Run(ctx, "pveversion", nil)
if err != nil {
if strings.Contains(err.Error(), "host key mismatch") {
_, fp, _ := sshx.Scan(ctx, o)
res.SSHMismatch, res.SSHFingerprint = true, fp
return fail(fmt.Errorf("SSH: node %s presents host key %s instead of the recorded %s", o.Host, fp, sshx.Fingerprint(key)))
}
return fail(fmt.Errorf("SSH: %w", err))
}
return w
if !strings.HasPrefix(strings.TrimSpace(string(out)), "pve-manager/") {
return fail(fmt.Errorf("SSH: %s is not a PVE node (pveversion: %q)", o.Host, strings.TrimSpace(string(out))))
}
if !has("/sdn", "SDN.Allocate") && !has("/sdn/zones", "SDN.Allocate") {
w = append(w, "SDN.Allocate absent sur /sdn : la synchro SDN échouera")
}
if dcOptions && !has("/", "Sys.Modify") {
w = append(w, "Sys.Modify absent sur / : les options datacenter ne pourront pas être écrites")
}
if !has("/", "Sys.Audit") {
w = append(w, "Sys.Audit absent sur / : options datacenter et versions illisibles")
}
return w
res.SSH, res.OK = true, true
return res
}
// nodes converts the versions read by the PVE client.
internal/sshx/sshx.go
+176
View File
@@ -0,0 +1,176 @@
// Package sshx runs commands on a PVE node over SSH as root, for what the API
// does not cover (the password hashes of the pve realm in /etc/pve/priv/shadow.cfg).
// The host key is pinned: the first connection records it (trust on first use),
// the next ones refuse any other key.
package sshx
import (
"bytes"
"context"
"errors"
"fmt"
"net"
"strconv"
"strings"
"time"
"golang.org/x/crypto/ssh"
)
// Options of an SSH access.
type Options struct {
Host string // node address
Port int // 22 by default
User string // root by default
Password string // never logged
HostKey string // pinned host key, authorized_keys format; empty: refused (see Scan)
Timeout time.Duration
}
// Client runs commands on one node. Each Run opens its own connection: the
// synchronizations are rare, a connection pool would only hold sessions open.
type Client struct{ o Options }
// New checks the options.
func New(o Options) (*Client, error) {
if o.Host == "" {
return nil, errors.New("ssh: host required")
}
if o.Password == "" {
return nil, errors.New("ssh: password required")
}
if o.HostKey == "" {
return nil, errors.New("ssh: host key not recorded yet, test the cluster first")
}
if _, _, _, _, err := ssh.ParseAuthorizedKey([]byte(o.HostKey)); err != nil {
return nil, fmt.Errorf("ssh: recorded host key: %w", err)
}
if o.Port == 0 {
o.Port = 22
}
if o.User == "" {
o.User = "root"
}
if o.Timeout == 0 {
o.Timeout = 15 * time.Second
}
return &Client{o}, nil
}
func (c *Client) addr() string { return net.JoinHostPort(c.o.Host, strconv.Itoa(c.o.Port)) }
// Run executes cmd and returns its standard output. in is written on the
// standard input (secrets are never put on the command line).
func (c *Client) Run(ctx context.Context, cmd string, in []byte) ([]byte, error) {
want, _, _, _, _ := ssh.ParseAuthorizedKey([]byte(c.o.HostKey))
cfg := &ssh.ClientConfig{
User: c.o.User,
Auth: auth(c.o.Password),
HostKeyCallback: ssh.FixedHostKey(want),
Timeout: c.o.Timeout,
}
conn, err := dial(ctx, c.addr(), cfg)
if err != nil {
return nil, err
}
defer conn.Close()
sess, err := conn.NewSession()
if err != nil {
return nil, fmt.Errorf("ssh %s: %w", c.addr(), err)
}
defer sess.Close()
var out, stderr bytes.Buffer
sess.Stdout, sess.Stderr = &out, &stderr
if in != nil {
sess.Stdin = bytes.NewReader(in)
}
done := make(chan error, 1)
go func() { done <- sess.Run(cmd) }()
select {
case err = <-done:
case <-ctx.Done():
conn.Close()
return nil, ctx.Err()
}
if err != nil {
msg := strings.TrimSpace(stderr.String())
if msg == "" {
msg = err.Error()
}
return out.Bytes(), fmt.Errorf("ssh %s: %s", c.addr(), msg)
}
return out.Bytes(), nil
}
// auth: password, and keyboard-interactive answering the password (PAM).
func auth(password string) []ssh.AuthMethod {
return []ssh.AuthMethod{
ssh.Password(password),
ssh.KeyboardInteractive(func(_, _ string, qs []string, _ []bool) ([]string, error) {
a := make([]string, len(qs))
for i := range a {
a[i] = password
}
return a, nil
}),
}
}
func dial(ctx context.Context, addr string, cfg *ssh.ClientConfig) (*ssh.Client, error) {
d := net.Dialer{Timeout: cfg.Timeout}
nc, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return nil, fmt.Errorf("ssh %s: %w", addr, err)
}
cc, chans, reqs, err := ssh.NewClientConn(nc, addr, cfg)
if err != nil {
nc.Close()
return nil, fmt.Errorf("ssh %s: %w", addr, err)
}
return ssh.NewClient(cc, chans, reqs), nil
}
// Scan connects without a pinned key and returns the key the node presents
// (authorized_keys format) and its SHA256 fingerprint, after checking that the
// password is accepted. The caller records the key (trust on first use).
func Scan(ctx context.Context, o Options) (key, fingerprint string, err error) {
if o.Port == 0 {
o.Port = 22
}
if o.User == "" {
o.User = "root"
}
if o.Timeout == 0 {
o.Timeout = 15 * time.Second
}
var seen ssh.PublicKey
cfg := &ssh.ClientConfig{
User: o.User,
Auth: auth(o.Password),
HostKeyCallback: func(_ string, _ net.Addr, k ssh.PublicKey) error {
seen = k
return nil
},
Timeout: o.Timeout,
}
addr := net.JoinHostPort(o.Host, strconv.Itoa(o.Port))
conn, err := dial(ctx, addr, cfg)
if seen != nil {
key = strings.TrimSpace(string(ssh.MarshalAuthorizedKey(seen)))
fingerprint = ssh.FingerprintSHA256(seen)
}
if err != nil {
return key, fingerprint, err
}
conn.Close()
return key, fingerprint, nil
}
// Fingerprint returns the SHA256 fingerprint of a recorded key ("" if invalid).
func Fingerprint(key string) string {
k, _, _, _, err := ssh.ParseAuthorizedKey([]byte(key))
if err != nil {
return ""
}
return ssh.FingerprintSHA256(k)
}
internal/store/crypto.go
+3 -10
View File
@@ -18,7 +18,7 @@ import (
"strings"
)
// Box encrypts the token secrets at rest (AES-256-GCM, random nonce, version byte).
// Box encrypts the cluster passwords at rest (AES-256-GCM, random nonce, version byte).
type Box struct{ aead cipher.AEAD }
const boxVersion = 1
@@ -65,7 +65,7 @@ func (b *Box) Open(blob []byte) ([]byte, error) {
return plain, nil
}
// LoadKey returns the encryption key of the token secrets, in this order:
// LoadKey returns the encryption key of the cluster passwords, in this order:
// CONFSYNC_SECRET_KEY (64 hex characters or 44 base64 characters),
// CONFSYNC_SECRET_KEY_FILE (same content, docker secret), or <dataDir>/secret.key,
// created on first start. The last case keeps the key next to the database: fine for
@@ -96,7 +96,7 @@ func LoadKey(dataDir string) ([]byte, string, error) {
if err := os.WriteFile(path, []byte(hex.EncodeToString(k)+"\n"), 0o600); err != nil {
return nil, "", fmt.Errorf("cannot save the encryption key: %w", err)
}
log.Printf("encryption key created in %s: keep a copy, the token secrets cannot be read without it", path)
log.Printf("encryption key created in %s: keep a copy, the cluster passwords cannot be read without it", path)
return k, path, nil
}
@@ -149,10 +149,3 @@ func CheckPassword(hash, pass string) bool {
got, err := pbkdf2.Key(sha256.New, pass, salt, iter, len(want))
return err == nil && subtle.ConstantTimeCompare(got, want) == 1
}
// RandomPassword returns a password for the first admin account.
func RandomPassword() string {
b := make([]byte, 12)
_, _ = rand.Read(b)
return base64.RawURLEncoding.EncodeToString(b)
}
internal/store/nodes.go
+433
View File
@@ -0,0 +1,433 @@
package store
import (
"context"
"database/sql"
"encoding/json"
"errors"
"fmt"
"strings"
"time"
"pve-confsync/internal/config"
"pve-confsync/internal/nodesync"
"pve-confsync/internal/sshx"
)
// nodeSchema: the nodes of the clusters (node replication) and the profiles.
// A node is reached over SSH as root at its address in the cluster (GET
// /cluster/status) or at an address given by the administrator, with the
// root password of its cluster or its own.
const nodeSchema = `
CREATE TABLE IF NOT EXISTS nodes (
cluster TEXT NOT NULL,
node TEXT NOT NULL,
ip TEXT NOT NULL DEFAULT '',
ssh_host TEXT NOT NULL DEFAULT '',
ssh_port INTEGER NOT NULL DEFAULT 22,
password BLOB,
ssh_hostkey TEXT NOT NULL DEFAULT '',
last_check TEXT NOT NULL DEFAULT '',
PRIMARY KEY (cluster, node)
);
CREATE TABLE IF NOT EXISTS node_profiles (
name TEXT PRIMARY KEY,
position INTEGER NOT NULL DEFAULT 0,
body TEXT NOT NULL,
updated_at INTEGER NOT NULL
);
`
// Node is a node as stored. The password is never returned to the UI.
type Node struct {
Key string `json:"key"` // "cluster/node"
Cluster string `json:"cluster"`
Node string `json:"node"`
IP string `json:"ip"` // address in the cluster
SSHHost string `json:"sshHost"` // override (empty: IP)
SSHPort int `json:"sshPort"`
Password string `json:"password,omitempty"` // write only (empty: the cluster's)
HasPassword bool `json:"hasPassword"`
SSHFingerprint string `json:"sshFingerprint"`
ClusterRole string `json:"clusterRole"`
LastCheck *Check `json:"lastCheck,omitempty"`
}
// NodeKey is the identity of a node in the profiles.
func NodeKey(cluster, node string) string { return cluster + "/" + node }
const nodeCols = `n.cluster, n.node, n.ip, n.ssh_host, n.ssh_port, n.password, n.ssh_hostkey, n.last_check, c.role`
func scanNode(scan func(...any) error) (Node, []byte, string, error) {
var n Node
var blob []byte
var key, check string
err := scan(&n.Cluster, &n.Node, &n.IP, &n.SSHHost, &n.SSHPort, &blob, &key, &check, &n.ClusterRole)
if err != nil {
return n, nil, "", err
}
n.Key = NodeKey(n.Cluster, n.Node)
n.HasPassword = len(blob) > 0
n.SSHFingerprint = sshx.Fingerprint(key)
if check != "" {
n.LastCheck = &Check{}
_ = json.Unmarshal([]byte(check), n.LastCheck)
}
return n, blob, key, nil
}
// Nodes lists the known nodes, by cluster then name.
func (s *Store) Nodes(ctx context.Context) ([]Node, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+nodeCols+` FROM nodes n JOIN clusters c ON c.name = n.cluster
ORDER BY c.role = 'primary' DESC, c.position, n.cluster, n.node`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []Node{}
for rows.Next() {
n, _, _, err := scanNode(rows.Scan)
if err != nil {
return nil, err
}
out = append(out, n)
}
return out, rows.Err()
}
// Discovered is a node read in GET /cluster/status.
type Discovered struct {
Name string
IP string
}
// SetNodes records the nodes of a cluster: new ones are added, the address of
// the known ones is updated (their pinned host key is forgotten when the SSH
// address changes), the nodes gone from the cluster are removed.
func (s *Store) SetNodes(ctx context.Context, cluster string, nodes []Discovered) error {
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
keep := map[string]bool{}
for _, d := range nodes {
keep[d.Name] = true
var ip, host string
err := tx.QueryRowContext(ctx, `SELECT ip, ssh_host FROM nodes WHERE cluster = ? AND node = ?`, cluster, d.Name).Scan(&ip, &host)
switch {
case errors.Is(err, sql.ErrNoRows):
_, err = tx.ExecContext(ctx, `INSERT INTO nodes (cluster, node, ip) VALUES (?,?,?)`, cluster, d.Name, d.IP)
case err == nil && ip != d.IP:
q := `UPDATE nodes SET ip = ? WHERE cluster = ? AND node = ?`
if host == "" {
q = `UPDATE nodes SET ip = ?, ssh_hostkey = '' WHERE cluster = ? AND node = ?`
}
_, err = tx.ExecContext(ctx, q, d.IP, cluster, d.Name)
}
if err != nil {
return err
}
}
rows, err := tx.QueryContext(ctx, `SELECT node FROM nodes WHERE cluster = ?`, cluster)
if err != nil {
return err
}
var gone []string
for rows.Next() {
var n string
if err := rows.Scan(&n); err != nil {
rows.Close()
return err
}
if !keep[n] {
gone = append(gone, n)
}
}
rows.Close()
for _, n := range gone {
if _, err := tx.ExecContext(ctx, `DELETE FROM nodes WHERE cluster = ? AND node = ?`, cluster, n); err != nil {
return err
}
}
return tx.Commit()
}
// SaveNode sets the SSH access of a node: address (empty: its address in the
// cluster), port, password (empty: unchanged; clear: back to the password
// of the cluster). A new address forgets the pinned host key.
func (s *Store) SaveNode(ctx context.Context, cluster, node, host string, port int, password string, clear bool) error {
if port == 0 {
port = 22
}
if port < 1 || port > 65535 {
return errors.New("ssh port: 1-65535 expected")
}
var oldHost string
var oldPort int
err := s.db.QueryRowContext(ctx, `SELECT ssh_host, ssh_port FROM nodes WHERE cluster = ? AND node = ?`, cluster, node).Scan(&oldHost, &oldPort)
if errors.Is(err, sql.ErrNoRows) {
return ErrNotFound
}
if err != nil {
return err
}
q := `UPDATE nodes SET ssh_host = ?, ssh_port = ?`
args := []any{strings.TrimSpace(host), port}
if oldHost != strings.TrimSpace(host) || oldPort != port {
q += `, ssh_hostkey = ''`
}
switch {
case clear:
q += `, password = NULL`
case password != "":
blob, err := s.box.Seal([]byte(password))
if err != nil {
return err
}
q += `, password = ?`
args = append(args, blob)
}
_, err = s.db.ExecContext(ctx, q+` WHERE cluster = ? AND node = ?`, append(args, cluster, node)...)
return err
}
// PinNode records the SSH host key seen on the first connection (empty: left
// unchanged) and the result of the test.
func (s *Store) PinNode(ctx context.Context, cluster, node, hostKey string, check *Check) error {
var chk []byte
if check != nil {
chk, _ = json.Marshal(check)
}
_, err := s.db.ExecContext(ctx, `UPDATE nodes SET
ssh_hostkey = CASE WHEN ? <> '' THEN ? ELSE ssh_hostkey END,
last_check = CASE WHEN ? <> '' THEN ? ELSE last_check END
WHERE cluster = ? AND node = ?`, hostKey, hostKey, string(chk), string(chk), cluster, node)
return err
}
// ForgetNode clears the pinned SSH host key of a node.
func (s *Store) ForgetNode(ctx context.Context, cluster, node string) error {
res, err := s.db.ExecContext(ctx, `UPDATE nodes SET ssh_hostkey = '' WHERE cluster = ? AND node = ?`, cluster, node)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
// NodeAccess is the decrypted access to one node: the connection of its
// cluster (API) and SSH as root on the node.
type NodeAccess struct {
Node
Cluster config.Cluster
SSH sshx.Options
}
// NodeAccesses returns the access to every node of the clusters not disabled.
func (s *Store) NodeAccesses(ctx context.Context) (map[string]NodeAccess, error) {
rows, err := s.db.QueryContext(ctx, `SELECT `+nodeCols+` FROM nodes n JOIN clusters c ON c.name = n.cluster WHERE c.role <> 'disabled'`)
if err != nil {
return nil, err
}
type row struct {
n Node
blob []byte
key string
}
var list []row
for rows.Next() {
n, blob, key, err := scanNode(rows.Scan)
if err != nil {
rows.Close()
return nil, err
}
list = append(list, row{n, blob, key})
}
rows.Close()
out := map[string]NodeAccess{}
clusters := map[string]config.Cluster{}
for _, r := range list {
cc, ok := clusters[r.n.Cluster]
if !ok {
var err error
if _, cc, err = s.Cluster(ctx, r.n.Cluster); err != nil {
return nil, err
}
clusters[r.n.Cluster] = cc
}
a, err := s.access(r.n, r.blob, r.key, cc)
if err != nil {
return nil, err
}
out[r.n.Key] = a
}
return out, nil
}
// NodeAccessOf returns the access to one node.
func (s *Store) NodeAccessOf(ctx context.Context, cluster, node string) (NodeAccess, error) {
n, blob, key, err := scanNode(s.db.QueryRowContext(ctx, `SELECT `+nodeCols+` FROM nodes n JOIN clusters c ON c.name = n.cluster
WHERE n.cluster = ? AND n.node = ?`, cluster, node).Scan)
if errors.Is(err, sql.ErrNoRows) {
return NodeAccess{}, ErrNotFound
}
if err != nil {
return NodeAccess{}, err
}
_, cc, err := s.Cluster(ctx, cluster)
if err != nil {
return NodeAccess{}, err
}
return s.access(n, blob, key, cc)
}
func (s *Store) access(n Node, blob []byte, key string, cc config.Cluster) (NodeAccess, error) {
pass := cc.Password
if len(blob) > 0 {
p, err := s.box.Open(blob)
if err != nil {
return NodeAccess{}, fmt.Errorf("node %s: password: %w", n.Key, err)
}
pass = string(p)
}
host := n.SSHHost
if host == "" {
host = n.IP
}
cc.SSHHostKey = "" // the cluster SSH access is not used here
return NodeAccess{Node: n, Cluster: cc,
SSH: sshx.Options{Host: host, Port: n.SSHPort, User: "root", Password: pass, HostKey: key}}, nil
}
// ---- profiles ----
// Profiles returns the node replication profiles, in order.
func (s *Store) Profiles(ctx context.Context) ([]nodesync.Profile, error) {
rows, err := s.db.QueryContext(ctx, `SELECT name, position, body FROM node_profiles ORDER BY position, name`)
if err != nil {
return nil, err
}
defer rows.Close()
out := []nodesync.Profile{}
for rows.Next() {
var name, body string
var pos int
if err := rows.Scan(&name, &pos, &body); err != nil {
return nil, err
}
var p nodesync.Profile
if err := json.Unmarshal([]byte(body), &p); err != nil {
return nil, fmt.Errorf("profile %s: %w", name, err)
}
p.Name, p.Position = name, pos
out = append(out, p)
}
return out, rows.Err()
}
// Profile returns one profile.
func (s *Store) Profile(ctx context.Context, name string) (nodesync.Profile, error) {
ps, err := s.Profiles(ctx)
if err != nil {
return nodesync.Profile{}, err
}
for _, p := range ps {
if p.Name == name {
return p, nil
}
}
return nodesync.Profile{}, ErrNotFound
}
// SaveProfile creates or updates a profile; oldName renames it.
func (s *Store) SaveProfile(ctx context.Context, oldName string, p nodesync.Profile) error {
if err := p.Validate(); err != nil {
return err
}
if oldName == "" {
oldName = p.Name
}
body, _ := json.Marshal(p)
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
res, err := tx.ExecContext(ctx, `UPDATE node_profiles SET name = ?, position = ?, body = ?, updated_at = ? WHERE name = ?`,
p.Name, p.Position, string(body), time.Now().Unix(), oldName)
if err == nil {
if n, _ := res.RowsAffected(); n == 0 {
_, err = tx.ExecContext(ctx, `INSERT INTO node_profiles (name, position, body, updated_at) VALUES (?,?,?,?)`,
p.Name, p.Position, string(body), time.Now().Unix())
}
}
if err != nil {
if strings.Contains(err.Error(), "UNIQUE") {
return fmt.Errorf("profile %q already exists", p.Name)
}
return err
}
return tx.Commit()
}
// DeleteProfile removes a profile.
func (s *Store) DeleteProfile(ctx context.Context, name string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM node_profiles WHERE name = ?`, name)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
// renameClusterNodes follows a renamed cluster in the nodes and the profiles.
func renameClusterNodes(ctx context.Context, tx *sql.Tx, old, name string) error {
if _, err := tx.ExecContext(ctx, `UPDATE nodes SET cluster = ? WHERE cluster = ?`, name, old); err != nil {
return err
}
rows, err := tx.QueryContext(ctx, `SELECT name, body FROM node_profiles`)
if err != nil {
return err
}
bodies := map[string]string{}
for rows.Next() {
var n, b string
if err := rows.Scan(&n, &b); err != nil {
rows.Close()
return err
}
bodies[n] = b
}
rows.Close()
ren := func(k string) string {
if strings.HasPrefix(k, old+"/") {
return name + k[len(old):]
}
return k
}
for n, b := range bodies {
var p nodesync.Profile
if json.Unmarshal([]byte(b), &p) != nil {
continue
}
p.Master = ren(p.Master)
for i, t := range p.Targets {
p.Targets[i] = ren(t)
}
m := map[string]nodesync.TargetMap{}
for k, v := range p.Mapping {
m[ren(k)] = v
}
p.Mapping = m
nb, _ := json.Marshal(p)
if _, err := tx.ExecContext(ctx, `UPDATE node_profiles SET body = ? WHERE name = ?`, string(nb), n); err != nil {
return err
}
}
return nil
}
internal/store/nodes_test.go
+66
View File
@@ -0,0 +1,66 @@
package store
import (
"context"
"path/filepath"
"testing"
"pve-confsync/internal/nodesync"
)
func TestNodesAndProfiles(t *testing.T) {
ctx := context.Background()
st, err := Open(filepath.Join(t.TempDir(), "t.db"), testKey(1))
if err != nil {
t.Fatal(err)
}
if err := st.SaveCluster(ctx, "", Cluster{Name: "lab", Role: ClusterPrimary, URL: "https://lab:8006", Password: "cluster-pass"}); err != nil {
t.Fatal(err)
}
if err := st.SetNodes(ctx, "lab", []Discovered{{"pve1", "10.0.0.11"}, {"pve2", "10.0.0.12"}}); err != nil {
t.Fatal(err)
}
if err := st.PinNode(ctx, "lab", "pve2", "ssh-ed25519 AAAA", nil); err != nil {
t.Fatal(err)
}
// the cluster password by default, the node's own one once set
a, err := st.NodeAccessOf(ctx, "lab", "pve2")
if err != nil || a.SSH.Password != "cluster-pass" || a.SSH.Host != "10.0.0.12" || a.SSH.HostKey == "" {
t.Fatalf("%+v %v", a.SSH, err)
}
if err := st.SaveNode(ctx, "lab", "pve2", "192.168.0.12", 22, "own-pass", false); err != nil {
t.Fatal(err)
}
a, _ = st.NodeAccessOf(ctx, "lab", "pve2")
if a.SSH.Password != "own-pass" || a.SSH.Host != "192.168.0.12" || a.SSH.HostKey != "" {
t.Fatalf("new address must forget the host key: %+v", a.SSH)
}
// a node gone from the cluster is removed
if err := st.SetNodes(ctx, "lab", []Discovered{{"pve1", "10.0.0.11"}}); err != nil {
t.Fatal(err)
}
if ns, _ := st.Nodes(ctx); len(ns) != 1 || ns[0].Key != "lab/pve1" {
t.Fatalf("nodes: %+v", ns)
}
p := nodesync.Profile{Name: "net", Master: "lab/pve1", Targets: []string{"lab/pve2"}, Items: []string{"network"},
Mapping: map[string]nodesync.TargetMap{"lab/pve2": {NICs: map[string]string{"eno1": "enp1s0"}}}}
if err := st.SaveProfile(ctx, "", p); err != nil {
t.Fatal(err)
}
if err := st.SaveProfile(ctx, "", nodesync.Profile{Name: "bad", Master: "lab/pve1", Targets: []string{"lab/pve1"}, Items: []string{"network"}}); err == nil {
t.Error("master as target accepted")
}
// renaming the cluster follows in the nodes and the profiles
c, _, _ := st.Cluster(ctx, "lab")
c.Name = "rennes"
if err := st.SaveCluster(ctx, "lab", c); err != nil {
t.Fatal(err)
}
got, err := st.Profile(ctx, "net")
if err != nil || got.Master != "rennes/pve1" || got.Targets[0] != "rennes/pve2" || got.Mapping["rennes/pve2"].NICs["eno1"] != "enp1s0" {
t.Fatalf("profile after rename: %+v %v", got, err)
}
if ns, _ := st.Nodes(ctx); len(ns) != 1 || ns[0].Key != "rennes/pve1" {
t.Fatalf("nodes after rename: %+v", ns)
}
}
internal/store/store.go
+306 -168
View File
@@ -1,6 +1,7 @@
// Package store keeps the state of the web UI in SQLite: clusters (token
// secrets encrypted with AES-256-GCM), synchronization settings, per client
// mappings and policies, users, execution journal and audit trail.
// Package store keeps the state of the web UI in SQLite: clusters (root
// passwords encrypted with AES-256-GCM, pinned TLS certificates and SSH host
// keys), synchronization settings, per target mappings and policies, users,
// execution journal and audit trail.
package store
import (
@@ -9,6 +10,7 @@ import (
"encoding/json"
"errors"
"fmt"
"log"
"strings"
"time"
@@ -17,6 +19,7 @@ import (
"pve-confsync/internal/adapter"
"pve-confsync/internal/config"
"pve-confsync/internal/model"
"pve-confsync/internal/sshx"
)
// ErrNotFound is returned for an unknown cluster, user or run.
@@ -28,20 +31,29 @@ type Store struct {
box *Box
}
// schemaVersion 2: clusters reached with the root credentials (API + SSH),
// roles primary / target / disabled. Version 1 (API tokens) is not migrated:
// its clusters table is dropped, the clusters are added again from the UI.
const schemaVersion = 2
const schema = `
CREATE TABLE IF NOT EXISTS clusters (
name TEXT PRIMARY KEY,
reference INTEGER NOT NULL DEFAULT 0,
role TEXT NOT NULL DEFAULT 'target',
position INTEGER NOT NULL DEFAULT 0,
url TEXT NOT NULL,
token_id TEXT NOT NULL,
secret BLOB,
api_user TEXT NOT NULL DEFAULT 'root@pam',
password BLOB,
tls TEXT NOT NULL DEFAULT 'fp',
fingerprint TEXT NOT NULL DEFAULT '',
ca_pem TEXT NOT NULL DEFAULT '',
insecure INTEGER NOT NULL DEFAULT 0,
timeout TEXT NOT NULL DEFAULT '',
ssh_host TEXT NOT NULL DEFAULT '',
ssh_port INTEGER NOT NULL DEFAULT 22,
ssh_hostkey TEXT NOT NULL DEFAULT '',
mapping TEXT NOT NULL DEFAULT '{}',
policies TEXT NOT NULL DEFAULT '{}',
last_check TEXT NOT NULL DEFAULT '',
updated_at INTEGER NOT NULL
);
CREATE TABLE IF NOT EXISTS settings (
@@ -96,7 +108,11 @@ func Open(path string, key []byte) (*Store, error) {
return nil, err
}
db.SetMaxOpenConns(1) // one writer; SQLite serializes anyway
if _, err := db.Exec(schema); err != nil {
if err := migrate(db); err != nil {
db.Close()
return nil, fmt.Errorf("%s: %w", path, err)
}
if _, err := db.Exec(schema + nodeSchema); err != nil {
db.Close()
return nil, fmt.Errorf("%s: %w", path, err)
}
@@ -108,6 +124,29 @@ func Open(path string, key []byte) (*Store, error) {
return s, nil
}
// migrate drops the clusters of the first schema (API tokens, not reusable).
func migrate(db *sql.DB) error {
var v int
if err := db.QueryRow(`PRAGMA user_version`).Scan(&v); err != nil {
return err
}
if v >= schemaVersion {
return nil
}
var old int
if err := db.QueryRow(`SELECT count(*) FROM pragma_table_info('clusters') WHERE name = 'token_id'`).Scan(&old); err != nil {
return err
}
if old > 0 {
log.Printf("database: clusters of the previous version (API tokens) dropped, add them again with their root credentials")
if _, err := db.Exec(`DROP TABLE clusters; DROP TABLE IF EXISTS settings`); err != nil {
return err
}
}
_, err := db.Exec(fmt.Sprintf(`PRAGMA user_version = %d`, schemaVersion))
return err
}
// Close closes the database.
func (s *Store) Close() error { return s.db.Close() }
@@ -115,7 +154,7 @@ func (s *Store) Close() error { return s.db.Close() }
// (lost or changed CONFSYNC_SECRET_KEY), instead of failing later on each cluster.
func (s *Store) checkKey() error {
var blob []byte
err := s.db.QueryRow(`SELECT secret FROM clusters WHERE secret IS NOT NULL AND length(secret) > 0 LIMIT 1`).Scan(&blob)
err := s.db.QueryRow(`SELECT password FROM clusters WHERE password IS NOT NULL AND length(password) > 0 LIMIT 1`).Scan(&blob)
if errors.Is(err, sql.ErrNoRows) {
return nil
}
@@ -123,67 +162,176 @@ func (s *Store) checkKey() error {
return err
}
if _, err := s.box.Open(blob); err != nil {
return errors.New("the encryption key does not decrypt the stored token secrets (CONFSYNC_SECRET_KEY changed?)")
return errors.New("the encryption key does not decrypt the stored passwords (CONFSYNC_SECRET_KEY changed?)")
}
return nil
}
// ---- clusters ----
// Cluster is a cluster as stored. The secret is never returned to the UI:
// HasSecret only says whether one is set.
// Cluster roles: one primary (the source, never written), targets (replicated)
// and disabled clusters (kept, not read).
const (
ClusterPrimary = "primary"
ClusterTarget = "target"
ClusterDisabled = "disabled"
)
// TLS modes of a cluster.
const (
TLSPin = "fp" // pinned certificate (recorded on the first connection when empty)
TLSCA = "ca" // CA bundle
TLSSystem = "sys" // system authorities
TLSInsecure = "insecure" // no verification (lab only)
)
// Check is the result of the last connection test of a cluster.
type Check struct {
At time.Time `json:"at"`
OK bool `json:"ok"`
Error string `json:"error,omitempty"`
Warnings []string `json:"warnings,omitempty"`
Version string `json:"version,omitempty"`
Nodes []string `json:"nodes,omitempty"`
API bool `json:"api"`
SSH bool `json:"ssh"`
}
// Cluster is a cluster as stored. The password is never returned to the UI:
// HasPassword only says whether one is set.
type Cluster struct {
Name string `json:"name"`
Reference bool `json:"reference"`
Role string `json:"role"`
Position int `json:"position"`
URL string `json:"url"`
TokenID string `json:"tokenId"`
Secret string `json:"secret,omitempty"` // write only
HasSecret bool `json:"hasSecret"`
User string `json:"user"`
Password string `json:"password,omitempty"` // write only
HasPassword bool `json:"hasPassword"`
TLS string `json:"tls"`
Fingerprint string `json:"fingerprint"`
CAPEM string `json:"caPem"`
Insecure bool `json:"insecure"`
Timeout string `json:"timeout"`
SSHHost string `json:"sshHost"`
SSHPort int `json:"sshPort"`
SSHHostKey string `json:"sshHostKey"` // read only (pinned by the connection test)
SSHFingerprint string `json:"sshFingerprint"` // read only
Mapping map[model.Kind]map[string]map[string]any `json:"mapping"`
Policies map[model.Kind]map[string]adapter.Policy `json:"policies"`
LastCheck *Check `json:"lastCheck,omitempty"`
UpdatedAt time.Time `json:"updatedAt"`
}
// Clusters returns every cluster, reference first, without secrets.
const clusterCols = `name, role, position, url, api_user, password, tls, fingerprint, ca_pem, timeout,
ssh_host, ssh_port, ssh_hostkey, mapping, policies, last_check, updated_at`
func scanCluster(scan func(...any) error) (Cluster, []byte, error) {
var c Cluster
var blob []byte
var mapping, policies, check string
var updated int64
err := scan(&c.Name, &c.Role, &c.Position, &c.URL, &c.User, &blob, &c.TLS, &c.Fingerprint, &c.CAPEM, &c.Timeout,
&c.SSHHost, &c.SSHPort, &c.SSHHostKey, &mapping, &policies, &check, &updated)
if err != nil {
return c, nil, err
}
c.HasPassword = len(blob) > 0
c.SSHFingerprint = sshx.Fingerprint(c.SSHHostKey)
c.UpdatedAt = time.Unix(updated, 0).UTC()
_ = json.Unmarshal([]byte(mapping), &c.Mapping)
_ = json.Unmarshal([]byte(policies), &c.Policies)
if check != "" {
c.LastCheck = &Check{}
_ = json.Unmarshal([]byte(check), c.LastCheck)
}
return c, blob, nil
}
// Clusters returns every cluster, primary first, without passwords.
func (s *Store) Clusters(ctx context.Context) ([]Cluster, error) {
rows, err := s.db.QueryContext(ctx, `SELECT name, reference, position, url, token_id, length(secret) > 0,
fingerprint, ca_pem, insecure, timeout, mapping, policies, updated_at
FROM clusters ORDER BY reference DESC, position, name`)
rows, err := s.db.QueryContext(ctx, `SELECT `+clusterCols+` FROM clusters
ORDER BY role = 'primary' DESC, position, name`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []Cluster
for rows.Next() {
var c Cluster
var hasSecret sql.NullBool
var mapping, policies string
var updated int64
if err := rows.Scan(&c.Name, &c.Reference, &c.Position, &c.URL, &c.TokenID, &hasSecret,
&c.Fingerprint, &c.CAPEM, &c.Insecure, &c.Timeout, &mapping, &policies, &updated); err != nil {
c, _, err := scanCluster(rows.Scan)
if err != nil {
return nil, err
}
c.HasSecret = hasSecret.Bool
c.UpdatedAt = time.Unix(updated, 0).UTC()
_ = json.Unmarshal([]byte(mapping), &c.Mapping)
_ = json.Unmarshal([]byte(policies), &c.Policies)
out = append(out, c)
}
return out, rows.Err()
}
// SaveCluster creates or updates a cluster. An empty Secret keeps the stored one.
// oldName renames a cluster (empty: same name). Setting Reference on a cluster
// makes it the only reference.
// Cluster returns one cluster with its decrypted connection (for a test).
func (s *Store) Cluster(ctx context.Context, name string) (Cluster, config.Cluster, error) {
c, blob, err := scanCluster(s.db.QueryRowContext(ctx, `SELECT `+clusterCols+` FROM clusters WHERE name = ?`, name).Scan)
if errors.Is(err, sql.ErrNoRows) {
return c, config.Cluster{}, ErrNotFound
}
if err != nil {
return c, config.Cluster{}, err
}
cc, err := s.connection(c, blob)
return c, cc, err
}
func (s *Store) connection(c Cluster, blob []byte) (config.Cluster, error) {
pass, err := s.box.Open(blob)
if err != nil {
return config.Cluster{}, fmt.Errorf("cluster %s: password: %w", c.Name, err)
}
cc := config.Cluster{Name: c.Name, URL: config.NormalizeURL(c.URL), User: c.User, Password: string(pass), Timeout: c.Timeout,
SSHHost: c.SSHHost, SSHPort: c.SSHPort, SSHHostKey: c.SSHHostKey}
switch c.TLS {
case TLSPin:
cc.Fingerprint = c.Fingerprint
case TLSCA:
cc.CAPEM = c.CAPEM
case TLSInsecure:
cc.Insecure = true
}
return cc, nil
}
// ValidClusterRole tells whether r is a cluster role.
func ValidClusterRole(r string) bool {
return r == ClusterPrimary || r == ClusterTarget || r == ClusterDisabled
}
// SaveCluster creates or updates a cluster. An empty Password keeps the stored
// one. oldName renames a cluster (empty: same name). A primary cluster makes
// the previous primary a target. The pinned TLS certificate and SSH host key
// are forgotten when the address they belong to changes.
func (s *Store) SaveCluster(ctx context.Context, oldName string, c Cluster) error {
c.Name = strings.TrimSpace(c.Name)
if c.Name == "" || c.URL == "" || c.TokenID == "" {
return errors.New("name, url and token id are required")
c.URL = config.NormalizeURL(c.URL)
if c.Name == "" || c.URL == "" {
return errors.New("name and url are required")
}
if c.User == "" {
c.User = "root@pam"
}
if c.Role == "" {
c.Role = ClusterTarget
}
if !ValidClusterRole(c.Role) {
return fmt.Errorf("role %q: primary, target or disabled expected", c.Role)
}
switch c.TLS {
case "":
c.TLS = TLSPin
case TLSPin, TLSCA, TLSSystem, TLSInsecure:
default:
return fmt.Errorf("tls %q: fp, ca, sys or insecure expected", c.TLS)
}
if c.TLS == TLSCA && strings.TrimSpace(c.CAPEM) == "" {
return errors.New("the CA certificate (PEM) is required")
}
if c.SSHPort == 0 {
c.SSHPort = 22
}
if oldName == "" {
oldName = c.Name
@@ -195,36 +343,47 @@ func (s *Store) SaveCluster(ctx context.Context, oldName string, c Cluster) erro
return err
}
defer tx.Rollback()
var exists bool
var secret []byte
err = tx.QueryRowContext(ctx, `SELECT 1, secret FROM clusters WHERE name = ?`, oldName).Scan(&exists, &secret)
old, blob, err := scanCluster(tx.QueryRowContext(ctx, `SELECT `+clusterCols+` FROM clusters WHERE name = ?`, oldName).Scan)
exists := err == nil
if err != nil && !errors.Is(err, sql.ErrNoRows) {
return err
}
if c.Secret != "" {
if secret, err = s.box.Seal([]byte(c.Secret)); err != nil {
if c.Password != "" {
if blob, err = s.box.Seal([]byte(c.Password)); err != nil {
return err
}
}
if len(secret) == 0 {
return errors.New("the token secret is required")
if len(blob) == 0 {
return errors.New("the root password is required")
}
if c.Reference {
if _, err := tx.ExecContext(ctx, `UPDATE clusters SET reference = 0`); err != nil {
hostKey := ""
if exists {
hostKey = old.SSHHostKey
if old.URL != c.URL && c.Fingerprint == old.Fingerprint {
c.Fingerprint = "" // another server: pinned again on the next test
}
if sshTarget(old) != sshTarget(c) {
hostKey = ""
}
}
if c.Role == ClusterPrimary {
if _, err := tx.ExecContext(ctx, `UPDATE clusters SET role = 'target' WHERE role = 'primary'`); err != nil {
return err
}
}
now := time.Now().Unix()
if exists {
_, err = tx.ExecContext(ctx, `UPDATE clusters SET name=?, reference=?, position=?, url=?, token_id=?, secret=?,
fingerprint=?, ca_pem=?, insecure=?, timeout=?, mapping=?, policies=?, updated_at=? WHERE name=?`,
c.Name, c.Reference, c.Position, c.URL, c.TokenID, secret, c.Fingerprint, c.CAPEM, c.Insecure, c.Timeout,
string(mapping), string(policies), now, oldName)
_, err = tx.ExecContext(ctx, `UPDATE clusters SET name=?, role=?, position=?, url=?, api_user=?, password=?, tls=?,
fingerprint=?, ca_pem=?, timeout=?, ssh_host=?, ssh_port=?, ssh_hostkey=?, mapping=?, policies=?, updated_at=?
WHERE name=?`,
c.Name, c.Role, c.Position, c.URL, c.User, blob, c.TLS, c.Fingerprint, c.CAPEM, c.Timeout,
c.SSHHost, c.SSHPort, hostKey, string(mapping), string(policies), now, oldName)
} else {
_, err = tx.ExecContext(ctx, `INSERT INTO clusters (name, reference, position, url, token_id, secret, fingerprint,
ca_pem, insecure, timeout, mapping, policies, updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)`,
c.Name, c.Reference, c.Position, c.URL, c.TokenID, secret, c.Fingerprint, c.CAPEM, c.Insecure, c.Timeout,
string(mapping), string(policies), now)
_, err = tx.ExecContext(ctx, `INSERT INTO clusters (name, role, position, url, api_user, password, tls, fingerprint,
ca_pem, timeout, ssh_host, ssh_port, ssh_hostkey, mapping, policies, updated_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`,
c.Name, c.Role, c.Position, c.URL, c.User, blob, c.TLS, c.Fingerprint, c.CAPEM, c.Timeout,
c.SSHHost, c.SSHPort, "", string(mapping), string(policies), now)
}
if err != nil {
if strings.Contains(err.Error(), "UNIQUE") {
@@ -232,9 +391,76 @@ func (s *Store) SaveCluster(ctx context.Context, oldName string, c Cluster) erro
}
return err
}
if exists && oldName != c.Name {
if err := renameClusterNodes(ctx, tx, oldName, c.Name); err != nil {
return err
}
}
return tx.Commit()
}
// sshTarget is the SSH endpoint of a cluster, to detect a change of node.
func sshTarget(c Cluster) string {
host := c.SSHHost
if host == "" {
host = c.URL
}
return fmt.Sprintf("%s|%d", host, c.SSHPort)
}
// SetRole changes the role of a cluster (a new primary demotes the previous one).
func (s *Store) SetRole(ctx context.Context, name, role string) error {
if !ValidClusterRole(role) {
return fmt.Errorf("role %q: primary, target or disabled expected", role)
}
tx, err := s.db.BeginTx(ctx, nil)
if err != nil {
return err
}
defer tx.Rollback()
if role == ClusterPrimary {
if _, err := tx.ExecContext(ctx, `UPDATE clusters SET role = 'target' WHERE role = 'primary' AND name <> ?`, name); err != nil {
return err
}
}
res, err := tx.ExecContext(ctx, `UPDATE clusters SET role = ?, updated_at = ? WHERE name = ?`, role, time.Now().Unix(), name)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return tx.Commit()
}
// Pin records the TLS fingerprint and/or the SSH host key seen on the first
// connection (empty values are left unchanged), and the result of the test.
func (s *Store) Pin(ctx context.Context, name, fingerprint, hostKey string, check *Check) error {
var chk []byte
if check != nil {
chk, _ = json.Marshal(check)
}
_, err := s.db.ExecContext(ctx, `UPDATE clusters SET
fingerprint = CASE WHEN ? <> '' THEN ? ELSE fingerprint END,
ssh_hostkey = CASE WHEN ? <> '' THEN ? ELSE ssh_hostkey END,
last_check = CASE WHEN ? <> '' THEN ? ELSE last_check END
WHERE name = ?`, fingerprint, fingerprint, hostKey, hostKey, string(chk), string(chk), name)
return err
}
// Forget clears the pinned TLS fingerprint and SSH host key (pinned again on the next test).
func (s *Store) Forget(ctx context.Context, name string) error {
res, err := s.db.ExecContext(ctx, `UPDATE clusters SET fingerprint = CASE WHEN tls = 'fp' THEN '' ELSE fingerprint END,
ssh_hostkey = '' WHERE name = ?`, name)
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
}
// DeleteCluster removes a cluster.
func (s *Store) DeleteCluster(ctx context.Context, name string) error {
res, err := s.db.ExecContext(ctx, `DELETE FROM clusters WHERE name = ?`, name)
@@ -244,7 +470,8 @@ func (s *Store) DeleteCluster(ctx context.Context, name string) error {
if n, _ := res.RowsAffected(); n == 0 {
return ErrNotFound
}
return nil
_, err = s.db.ExecContext(ctx, `DELETE FROM nodes WHERE cluster = ?`, name)
return err
}
func nonNil(m map[model.Kind]map[string]map[string]any) any {
@@ -288,6 +515,13 @@ func (s *Store) Settings(ctx context.Context) (Settings, error) {
if len(st.Kinds) == 0 {
st.Kinds = append(st.Kinds, model.Kinds...)
}
// empty maps, not null, for the web UI
if st.Objects == nil {
st.Objects = map[model.Kind]config.Selection{}
}
if st.Policies == nil {
st.Policies = map[model.Kind]map[string]adapter.Policy{}
}
return st, nil
}
@@ -304,43 +538,38 @@ func (s *Store) SaveSettings(ctx context.Context, st Settings) error {
// ---- configuration for the engine ----
// Config builds the engine configuration, secrets decrypted. It fails when no
// reference is defined.
// Config builds the engine configuration, passwords decrypted: the primary
// cluster and the targets (disabled clusters are left aside). It fails when no
// primary is defined.
func (s *Store) Config(ctx context.Context) (*config.Config, error) {
st, err := s.Settings(ctx)
if err != nil {
return nil, err
}
cfg := &config.Config{Kinds: st.Kinds, Objects: st.Objects, Policies: st.Policies, Prune: st.Prune}
rows, err := s.db.QueryContext(ctx, `SELECT name, reference, url, token_id, secret, fingerprint, ca_pem,
insecure, timeout, mapping, policies FROM clusters ORDER BY reference DESC, position, name`)
rows, err := s.db.QueryContext(ctx, `SELECT `+clusterCols+` FROM clusters ORDER BY position, name`)
if err != nil {
return nil, err
}
defer rows.Close()
hasRef := false
for rows.Next() {
var ref bool
var blob []byte
var mapping, policies string
var cl config.Cluster
if err := rows.Scan(&cl.Name, &ref, &cl.URL, &cl.TokenID, &blob, &cl.Fingerprint, &cl.CAPEM,
&cl.Insecure, &cl.Timeout, &mapping, &policies); err != nil {
c, blob, err := scanCluster(rows.Scan)
if err != nil {
return nil, err
}
secret, err := s.box.Open(blob)
if err != nil {
return nil, fmt.Errorf("cluster %s: token secret: %w", cl.Name, err)
}
cl.Secret = string(secret)
if ref {
cfg.Reference, hasRef = cl, true
if c.Role == ClusterDisabled {
continue
}
c := config.Client{Cluster: cl}
_ = json.Unmarshal([]byte(mapping), &c.Mapping)
_ = json.Unmarshal([]byte(policies), &c.Policies)
cfg.Clients = append(cfg.Clients, c)
cc, err := s.connection(c, blob)
if err != nil {
return nil, err
}
if c.Role == ClusterPrimary {
cfg.Reference, hasRef = cc, true
continue
}
cfg.Clients = append(cfg.Clients, config.Client{Cluster: cc, Mapping: c.Mapping, Policies: c.Policies})
}
if err := rows.Err(); err != nil {
return nil, err
@@ -351,96 +580,5 @@ func (s *Store) Config(ctx context.Context) (*config.Config, error) {
return cfg, cfg.Validate()
}
// ErrNoReference: nothing can be compared before a reference cluster is defined.
var ErrNoReference = errors.New("no reference cluster defined")
// ExportConfig returns the configuration as the CLI YAML file, without secrets:
// each cluster gets a token_secret_env placeholder to fill.
func (s *Store) ExportConfig(ctx context.Context) (*config.Config, error) {
st, err := s.Settings(ctx)
if err != nil {
return nil, err
}
cls, err := s.Clusters(ctx)
if err != nil {
return nil, err
}
cfg := &config.Config{Kinds: st.Kinds, Objects: st.Objects, Policies: st.Policies, Prune: st.Prune}
for _, c := range cls {
cl := config.Cluster{Name: c.Name, URL: c.URL, TokenID: c.TokenID, Fingerprint: c.Fingerprint,
Insecure: c.Insecure, Timeout: c.Timeout, TokenSecretEnv: EnvName(c.Name)}
if c.Reference {
cfg.Reference = cl
continue
}
cfg.Clients = append(cfg.Clients, config.Client{Cluster: cl, Mapping: c.Mapping, Policies: c.Policies})
}
return cfg, nil
}
// EnvName is the environment variable proposed for the secret of a cluster in an export.
func EnvName(cluster string) string {
var b strings.Builder
b.WriteString("PVE_")
for _, r := range strings.ToUpper(cluster) {
if (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') {
b.WriteRune(r)
} else {
b.WriteByte('_')
}
}
b.WriteString("_SECRET")
return b.String()
}
// ImportConfig loads a CLI configuration: settings are replaced, clusters are
// created or updated (the ones absent from the file are kept unless replace is
// set). Secrets are read from token_secret_env / token_secret_file when they
// are reachable from the server; otherwise the stored secret is kept.
// It returns warnings for the clusters left without a secret.
func (s *Store) ImportConfig(ctx context.Context, cfg *config.Config, replace bool) ([]string, error) {
var warns []string
st := Settings{Kinds: cfg.Kinds, Objects: cfg.Objects, Policies: cfg.Policies, Prune: cfg.Prune}
existing, err := s.Clusters(ctx)
if err != nil {
return nil, err
}
have := map[string]Cluster{}
for _, c := range existing {
have[c.Name] = c
}
save := func(cl config.Cluster, ref bool, pos int, mapping map[model.Kind]map[string]map[string]any, pol map[model.Kind]map[string]adapter.Policy) error {
c := Cluster{Name: cl.Name, Reference: ref, Position: pos, URL: cl.URL, TokenID: cl.TokenID,
Fingerprint: cl.Fingerprint, Insecure: cl.Insecure, Timeout: cl.Timeout, Mapping: mapping, Policies: pol}
if cl.CAFile != "" {
warns = append(warns, fmt.Sprintf("%s: ca_file %s not imported, paste the CA certificate in the cluster settings", cl.Name, cl.CAFile))
}
if sec, err := cl.ResolveSecret(); err == nil && sec != "" {
c.Secret = sec
} else if !have[cl.Name].HasSecret {
warns = append(warns, fmt.Sprintf("%s: token secret not readable from the server, cluster not imported (%v)", cl.Name, err))
return nil
}
return s.SaveCluster(ctx, "", c)
}
if err := save(cfg.Reference, true, 0, nil, nil); err != nil {
return warns, err
}
keep := map[string]bool{cfg.Reference.Name: true}
for i, c := range cfg.Clients {
keep[c.Name] = true
if err := save(c.Cluster, false, i+1, c.Mapping, c.Policies); err != nil {
return warns, err
}
}
if replace {
for name := range have {
if !keep[name] {
if err := s.DeleteCluster(ctx, name); err != nil {
return warns, err
}
}
}
}
return warns, s.SaveSettings(ctx, st)
}
// ErrNoReference: nothing can be compared before a primary cluster is chosen.
var ErrNoReference = errors.New("no primary cluster defined")
internal/store/store_test.go
+106 -14
View File
@@ -3,6 +3,7 @@ package store
import (
"bytes"
"context"
"database/sql"
"path/filepath"
"strings"
"testing"
@@ -10,30 +11,30 @@ import (
func testKey(b byte) []byte { return bytes.Repeat([]byte{b}, 32) }
func TestSecretsEncryptedAndKeyChecked(t *testing.T) {
func TestPasswordsEncryptedAndKeyChecked(t *testing.T) {
ctx := context.Background()
path := filepath.Join(t.TempDir(), "t.db")
st, err := Open(path, testKey(1))
if err != nil {
t.Fatal(err)
}
if err := st.SaveCluster(ctx, "", Cluster{Name: "ref", Reference: true, URL: "https://ref:8006", TokenID: "a@pve!ro", Secret: "s3cr3t-uuid"}); err != nil {
if err := st.SaveCluster(ctx, "", Cluster{Name: "ref", Role: ClusterPrimary, URL: "https://ref:8006", Password: "s3cr3t-root"}); err != nil {
t.Fatal(err)
}
var blob []byte
if err := st.db.QueryRowContext(ctx, `SELECT secret FROM clusters WHERE name = 'ref'`).Scan(&blob); err != nil {
if err := st.db.QueryRowContext(ctx, `SELECT password FROM clusters WHERE name = 'ref'`).Scan(&blob); err != nil {
t.Fatal(err)
}
if bytes.Contains(blob, []byte("s3cr3t")) {
t.Fatal("secret stored in clear")
t.Fatal("password stored in clear")
}
cls, _ := st.Clusters(ctx)
if len(cls) != 1 || cls[0].Secret != "" || !cls[0].HasSecret {
t.Fatalf("listing must hide the secret: %+v", cls)
if len(cls) != 1 || cls[0].Password != "" || !cls[0].HasPassword || cls[0].User != "root@pam" || cls[0].TLS != TLSPin {
t.Fatalf("listing must hide the password: %+v", cls)
}
// an empty secret on update keeps the stored one
// an empty password on update keeps the stored one
c := cls[0]
c.URL = "https://ref2:8006"
c.Timeout = "20s"
if err := st.SaveCluster(ctx, "ref", c); err != nil {
t.Fatal(err)
}
@@ -41,12 +42,8 @@ func TestSecretsEncryptedAndKeyChecked(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if s, _ := cfg.Reference.ResolveSecret(); s != "s3cr3t-uuid" {
t.Fatalf("secret after update: %q", s)
}
exp, _ := st.ExportConfig(ctx)
if exp.Reference.Secret != "" || exp.Reference.TokenSecretEnv != "PVE_REF_SECRET" {
t.Fatalf("export: %+v", exp.Reference)
if cfg.Reference.Password != "s3cr3t-root" || cfg.Reference.User != "root@pam" {
t.Fatalf("password after update: %+v", cfg.Reference)
}
st.Close()
if _, err := Open(path, testKey(2)); err == nil {
@@ -54,6 +51,74 @@ func TestSecretsEncryptedAndKeyChecked(t *testing.T) {
}
}
func TestRolesAndPins(t *testing.T) {
ctx := context.Background()
st, err := Open(filepath.Join(t.TempDir(), "t.db"), testKey(1))
if err != nil {
t.Fatal(err)
}
defer st.Close()
if _, err := st.Config(ctx); err != ErrNoReference {
t.Fatalf("empty database: %v", err)
}
for _, n := range []string{"a", "b", "c"} {
if err := st.SaveCluster(ctx, "", Cluster{Name: n, URL: "https://" + n + ":8006", Password: "pw"}); err != nil {
t.Fatal(err)
}
}
if err := st.SetRole(ctx, "a", ClusterPrimary); err != nil {
t.Fatal(err)
}
if err := st.SetRole(ctx, "c", ClusterDisabled); err != nil {
t.Fatal(err)
}
if err := st.SetRole(ctx, "b", ClusterPrimary); err != nil { // a becomes a target
t.Fatal(err)
}
cfg, err := st.Config(ctx)
if err != nil {
t.Fatal(err)
}
if cfg.Reference.Name != "b" || len(cfg.Clients) != 1 || cfg.Clients[0].Name != "a" {
t.Fatalf("primary %s, targets %+v", cfg.Reference.Name, cfg.Clients)
}
if err := st.Pin(ctx, "a", "AA:BB", "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOMqqnkVzrm0SdG6UOoqKLsabgH5C9okWi0dh2l9GKJl", &Check{OK: true}); err != nil {
t.Fatal(err)
}
_, cc, _ := st.Cluster(ctx, "a")
if cc.Fingerprint != "AA:BB" || cc.SSHHostKey == "" {
t.Fatalf("pins = %+v", cc)
}
// another address: the pins are forgotten
c, _, _ := st.Cluster(ctx, "a")
c.URL = "https://a2:8006"
if err := st.SaveCluster(ctx, "a", c); err != nil {
t.Fatal(err)
}
_, cc, _ = st.Cluster(ctx, "a")
if cc.Fingerprint != "" || cc.SSHHostKey != "" {
t.Fatalf("pins kept after an address change: %+v", cc)
}
}
func TestFirstAdmin(t *testing.T) {
ctx := context.Background()
st, err := Open(filepath.Join(t.TempDir(), "t.db"), testKey(1))
if err != nil {
t.Fatal(err)
}
defer st.Close()
if err := st.CreateFirstAdmin(ctx, "admin", "too-short"); err == nil {
t.Fatal("short password accepted")
}
if err := st.CreateFirstAdmin(ctx, "admin", "a-long-password"); err != nil {
t.Fatal(err)
}
if err := st.CreateFirstAdmin(ctx, "other", "a-long-password"); err != ErrSetupDone {
t.Fatalf("second setup: %v", err)
}
}
func TestUsers(t *testing.T) {
ctx := context.Background()
st, err := Open(filepath.Join(t.TempDir(), "t.db"), testKey(1))
@@ -99,3 +164,30 @@ func TestPasswordHash(t *testing.T) {
t.Fatal("hash check")
}
}
func TestMigrateFromTokens(t *testing.T) {
path := filepath.Join(t.TempDir(), "t.db")
db, err := sql.Open("sqlite", "file:"+path)
if err != nil {
t.Fatal(err)
}
if _, err := db.Exec(`CREATE TABLE clusters (name TEXT PRIMARY KEY, reference INTEGER, url TEXT, token_id TEXT, secret BLOB, updated_at INTEGER);
INSERT INTO clusters VALUES ('old', 1, 'https://x:8006', 'a@pve!t', x'01', 0);
CREATE TABLE users (name TEXT PRIMARY KEY, role TEXT NOT NULL, password TEXT NOT NULL, epoch INTEGER NOT NULL, created_at INTEGER NOT NULL, last_login INTEGER NOT NULL DEFAULT 0);
INSERT INTO users VALUES ('admin', 'admin', 'h', 1, 1, 0)`); err != nil {
t.Fatal(err)
}
db.Close()
st, err := Open(path, testKey(1))
if err != nil {
t.Fatal(err)
}
defer st.Close()
cls, err := st.Clusters(context.Background())
if err != nil || len(cls) != 0 {
t.Fatalf("clusters after migration: %v %v", cls, err)
}
if n, _ := st.CountUsers(context.Background()); n != 1 {
t.Fatal("users must be kept")
}
}
internal/store/users.go
+30
View File
@@ -180,3 +180,33 @@ func (s *Store) keepAnAdmin(ctx context.Context, name, newRole string) error {
}
return nil
}
// ErrSetupDone: the first administrator already exists.
var ErrSetupDone = errors.New("the first administrator already exists")
// CreateFirstAdmin creates the first administrator from the web page shown on an
// empty database. The insert only happens while the table is empty: two
// concurrent requests cannot both succeed.
func (s *Store) CreateFirstAdmin(ctx context.Context, name, pass string) error {
name = strings.TrimSpace(name)
if name == "" || strings.ContainsAny(name, " \t\r\n") {
return errors.New("invalid user name")
}
if len(pass) < 12 {
return errors.New("password: 12 characters minimum")
}
hash, err := HashPassword(pass)
if err != nil {
return err
}
now := time.Now()
res, err := s.db.ExecContext(ctx, `INSERT INTO users (name, role, password, epoch, created_at)
SELECT ?, 'admin', ?, ?, ? WHERE NOT EXISTS (SELECT 1 FROM users)`, name, hash, now.UnixNano(), now.Unix())
if err != nil {
return err
}
if n, _ := res.RowsAffected(); n == 0 {
return ErrSetupDone
}
return nil
}
web/app.js
+1124 -225
View File
File diff suppressed because it is too large. Load diff
web/index.html
+195 -68
View File
@@ -16,6 +16,20 @@
<span id="demoBadge" class="demo-badge" hidden data-i18n="demo">DÉMO</span>
</div>
<!-- the two replications: cluster to cluster (API), node to node (system files) -->
<div id="modeSw" class="seg mode-sw" role="radiogroup" data-i18n-aria="modeAriaSw">
<button type="button" role="radio" data-mode="cluster" data-i18n-title="modeClusterTitle">
<!-- Lucide "layers" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="m12.83 2.18a2 2 0 0 0-1.66 0L2.6 6.08a1 1 0 0 0 0 1.83l8.58 3.91a2 2 0 0 0 1.66 0l8.58-3.9a1 1 0 0 0 0-1.83Z"/><path d="m22 17.65-9.17 4.16a2 2 0 0 1-1.66 0L2 17.65"/><path d="m22 12.65-9.17 4.16a2 2 0 0 1-1.66 0L2 12.65"/></svg>
<span data-i18n="modeCluster">Clusters (API)</span>
</button>
<button type="button" role="radio" data-mode="node" data-i18n-title="modeNodeTitle">
<!-- Lucide "hard-drive" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M22 12H2"/><path d="M5.45 5.11 2 12v6a2 2 0 0 0 2 2h16a2 2 0 0 0 2-2v-6l-3.45-6.89A2 2 0 0 0 16.76 4H7.24a2 2 0 0 0-1.79 1.11z"/><path d="M6 16h.01M10 16h.01"/></svg>
<span data-i18n="modeNode">Nœuds (fichiers)</span>
</button>
</div>
<div class="search">
<svg class="ico" viewBox="0 0 24 24" aria-hidden="true"><circle cx="11" cy="11" r="7"/><path d="m20 20-3.5-3.5"/></svg>
<input id="q" type="search" autocomplete="off" spellcheck="false" data-i18n-ph="searchPh" data-i18n-aria="searchPh">
@@ -51,6 +65,16 @@
<svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="18" cy="18" r="3"/><circle cx="6" cy="6" r="3"/><path d="M13 6h3a2 2 0 0 1 2 2v7"/><path d="M11 18H8a2 2 0 0 1-2-2V9"/></svg>
<span data-i18n="viewDiff">Écarts</span>
</button>
<button type="button" role="radio" data-view="clusters">
<!-- Lucide "server" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01M6 18h.01"/></svg>
<span data-i18n="viewClusters">Clusters</span>
</button>
<button type="button" role="radio" data-view="nodes" hidden>
<!-- Lucide "server" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01M6 18h.01"/></svg>
<span data-i18n="viewNodes">Nœuds et profils</span>
</button>
<button type="button" role="radio" data-view="runs">
<!-- Lucide "history" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M3 12a9 9 0 1 0 9-9 9.75 9.75 0 0 0-6.74 2.74L3 8"/><path d="M3 3v5h5"/><path d="M12 7v5l4 2"/></svg>
@@ -62,12 +86,21 @@
<span data-i18n="viewAudit">Audit</span>
</button>
</div>
<select id="profileSel" data-diff data-node-only data-i18n-aria="profileAria"></select>
<select id="kindSel" data-diff data-i18n-aria="kindAria"></select>
<select id="statusSel" data-diff data-i18n-aria="statusAria"></select>
<select id="runModeSel" data-runs data-i18n-aria="modeAria" hidden></select>
<span class="spacer"></span>
<span id="count" class="muted"></span>
<div id="globalActs" class="global-acts" data-diff hidden>
<details id="targetMenu" class="menu right">
<summary>
<!-- Lucide "target" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><circle cx="12" cy="12" r="10"/><circle cx="12" cy="12" r="6"/><circle cx="12" cy="12" r="2"/></svg>
<span data-i18n="targetMenu">Cibles</span><span class="n" id="targetMenuN"></span>
</summary>
<div class="pop" id="targetMenuPop"></div>
</details>
<button id="dryAllBtn" class="btn tool" type="button">
<!-- Lucide "scan-search" (ISC license) -->
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M3 7V5a2 2 0 0 1 2-2h2M17 3h2a2 2 0 0 1 2 2v2M21 17v2a2 2 0 0 1-2 2h-2M7 21H5a2 2 0 0 1-2-2v-2"/><circle cx="12" cy="12" r="3"/><path d="m16 16-1.9-1.9"/></svg>
@@ -75,15 +108,15 @@
</button>
<button id="syncAllBtn" class="btn tool sync" type="button">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M3 12a9 9 0 0 1 15.7-6L21 8M21 3v5h-5M21 12a9 9 0 0 1-15.7 6L3 16M3 21v-5h5"/></svg>
<span data-i18n="syncAll">Tout synchroniser</span>
<span data-i18n="syncAll">Répliquer</span>
</button>
</div>
</section>
<div id="error" class="error-banner" hidden></div>
<div id="setup" class="setup" hidden>
<p data-i18n="setupText">Aucun cluster de référence n'est défini.</p>
<button id="setupBtn" class="btn primary" type="button" data-i18n="setupBtn">Ajouter les clusters</button>
<p id="setupText"></p>
<button id="setupBtn" class="btn primary" type="button"></button>
</div>
<!-- Differences: clusters, then the objects of the selected client -->
@@ -108,6 +141,31 @@
</div>
</section>
<!-- Clusters: connection (root), primary / targets, connection tests -->
<section id="clustersView" class="cl-view" hidden>
<div class="cl-tools">
<p class="muted small" data-i18n="clustersHelp"></p>
<button id="addCluster" class="btn tool primary" type="button" data-admin-only data-i18n="addCluster">+ Ajouter un cluster</button>
<button id="addBulk" class="btn tool" type="button" data-admin-only data-i18n="addBulk">+ Ajouter une liste</button>
<button id="testAll" class="btn tool" type="button" data-admin-only data-i18n="testAll">Tout tester</button>
</div>
<div id="clusterList" class="clist"></div>
</section>
<!-- Nodes (SSH access) and node replication profiles -->
<section id="nodesView" class="cl-view" hidden>
<div class="cl-tools">
<p class="muted small" data-i18n="nodesHelp"></p>
<button id="addProfile" class="btn tool primary" type="button" data-admin-only data-i18n="addProfile">+ Nouveau profil</button>
<button id="discoverBtn" class="btn tool" type="button" data-admin-only data-i18n="discover">Découvrir les nœuds</button>
<button id="testNodes" class="btn tool" type="button" data-admin-only data-i18n="testAll">Tout tester</button>
</div>
<h3 class="sec-title" data-i18n="profilesTitle">Profils de réplication</h3>
<div id="profileList" class="clist"></div>
<h3 class="sec-title" data-i18n="nodesTitle">Nœuds</h3>
<div id="nodeList" class="clist"></div>
</section>
<!-- Execution journal -->
<section id="runsView" hidden>
<div class="table runs">
@@ -159,6 +217,7 @@
<h2 id="planTitle"></h2>
<button class="icon-btn" type="button" data-close data-i18n-aria="close"><svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg></button>
</header>
<div id="planKinds" class="plan-kinds" hidden></div>
<div id="planBody" class="dlg-body"></div>
<footer>
<span id="planNote" class="muted small"></span>
@@ -166,7 +225,7 @@
<button id="planClose" class="btn ghost" type="button" data-close data-i18n="close">Fermer</button>
<button id="planApply" class="btn primary" type="button" hidden>
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M3 12a9 9 0 0 1 15.7-6L21 8M21 3v5h-5M21 12a9 9 0 0 1-15.7 6L3 16M3 21v-5h5"/></svg>
<span data-i18n="applyBtn">Synchroniser</span>
<span data-i18n="applyBtn">Répliquer</span>
</button>
</footer>
</div>
@@ -185,39 +244,46 @@
<input id="cName" class="inp" required autocomplete="off" spellcheck="false">
<span class="lbl" data-i18n="cRole">Rôle</span>
<div id="cRole" class="seg" role="radiogroup">
<button type="button" role="radio" data-v="client" data-i18n="roleClient">Client</button>
<button type="button" role="radio" data-v="ref" data-i18n="roleRef">Référence</button>
<button type="button" role="radio" data-v="primary" data-i18n="rolePrimary">Primaire</button>
<button type="button" role="radio" data-v="target" data-i18n="roleTarget">Cible</button>
<button type="button" role="radio" data-v="disabled" data-i18n="roleDisabled">Inactif</button>
</div>
<label for="cURL">URL</label>
<input id="cURL" class="inp mono" required placeholder="https://pve.example.org:8006" autocomplete="off" spellcheck="false">
<label for="cToken" data-i18n="cToken">Token</label>
<input id="cToken" class="inp mono" required placeholder="confsync@pve!rw" autocomplete="off" spellcheck="false">
<label for="cSecret" data-i18n="cSecret">Secret</label>
<input id="cSecret" class="inp mono" type="password" autocomplete="new-password" spellcheck="false">
<p class="muted small hint" id="cSecretHint"></p>
<input id="cURL" class="inp mono" required placeholder="https://pve1.example.org:8006" autocomplete="off" spellcheck="false">
<label for="cUser" data-i18n="cUser">Utilisateur</label>
<input id="cUser" class="inp mono" value="root@pam" readonly tabindex="-1">
<label for="cPassword" data-i18n="cPassword">Mot de passe root</label>
<input id="cPassword" class="inp mono" type="password" autocomplete="new-password" spellcheck="false">
<p class="muted small hint" id="cPasswordHint"></p>
<span class="lbl" data-i18n="cTLS">TLS</span>
<select id="cTLS" class="field">
<option value="fp" data-i18n="tlsFp">Empreinte SHA-256 du certificat</option>
<option value="fp" data-i18n="tlsFp">Empreinte du certificat (enregistrée au premier test)</option>
<option value="ca" data-i18n="tlsCa">Certificat d'autorité (PEM)</option>
<option value="sys" data-i18n="tlsSys">Autorités du système</option>
<option value="insecure" data-i18n="tlsInsecure">Sans vérification (labo uniquement)</option>
</select>
<label for="cFp" data-tls="fp" data-i18n="cFp">Empreinte</label>
<input id="cFp" class="inp mono" data-tls="fp" placeholder="AA:BB:…" autocomplete="off" spellcheck="false">
<input id="cFp" class="inp mono" data-tls="fp" data-i18n-ph="fpPh" autocomplete="off" spellcheck="false">
<label for="cCA" data-tls="ca">PEM</label>
<textarea id="cCA" class="inp" data-tls="ca" spellcheck="false" placeholder="-----BEGIN CERTIFICATE-----"></textarea>
<label for="cSSHHost" data-i18n="cSSHHost">Nœud SSH</label>
<input id="cSSHHost" class="inp mono" data-i18n-ph="sshHostPh" autocomplete="off" spellcheck="false">
<label for="cSSHPort" data-i18n="cSSHPort">Port SSH</label>
<input id="cSSHPort" class="inp mono" type="number" min="1" max="65535" placeholder="22">
<span class="lbl" data-i18n="cSSHKey">Clé d'hôte SSH</span>
<span id="cSSHKey" class="mono small muted"></span>
<label for="cTimeout" data-i18n="cTimeout">Délai</label>
<input id="cTimeout" class="inp mono" placeholder="30s" autocomplete="off">
</div>
<div id="cClientOnly">
<div class="sec-head mt"><h3 class="lbl" data-i18n="cMapping">Mapping (valeurs propres au client)</h3></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="cMapping">Mapping (valeurs propres à la cible)</h3></div>
<p class="muted small" data-i18n="cMappingHelp"></p>
<div class="erow-head map"><span data-i18n="colKind">Type</span><span data-i18n="colObject">Objet</span><span data-i18n="colField">Champ</span><span data-i18n="colValue">Valeur</span><span></span></div>
<div id="cMapRows" class="erows"></div>
<div class="toolbar"><button id="cMapAdd" class="btn tool" type="button" data-i18n="addRow">+ Ajouter</button></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="cPolicies">Politiques propres au client</h3></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="cPolicies">Politiques propres à la cible</h3></div>
<p class="muted small" data-i18n="cPoliciesHelp"></p>
<div class="erow-head pol"><span data-i18n="colKind">Type</span><span data-i18n="colField">Champ</span><span data-i18n="colPolicy">Politique</span><span></span></div>
<div id="cPolRows" class="erows"></div>
@@ -227,6 +293,115 @@
</div>
<footer>
<button id="cDelete" class="btn danger" type="button" data-i18n="delete">Supprimer</button>
<button id="cForget" class="btn ghost" type="button" data-i18n="forget">Oublier les empreintes</button>
<span class="sp"></span>
<button class="btn ghost" type="button" data-close data-i18n="cancel">Annuler</button>
<button class="btn primary" type="submit" data-i18n="save">Enregistrer</button>
</footer>
</form>
</dialog>
<!-- Several clusters at once (administrators) -->
<dialog id="bulkDlg" aria-labelledby="bulkTitle">
<form class="dlg" id="bulkForm" method="dialog">
<header>
<h2 id="bulkTitle" data-i18n="bulkTitle">Ajouter une liste de clusters</h2>
<button class="icon-btn" type="button" data-close data-i18n-aria="close"><svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg></button>
</header>
<div class="dlg-body">
<p class="muted small" data-i18n="bulkHelp"></p>
<textarea id="bulkText" class="inp" spellcheck="false" required placeholder="pve-rennes https://10.0.0.11:8006&#10;pve-nantes https://10.0.1.11:8006&#10;https://pve-brest.lab:8006"></textarea>
<div class="field-grid mt-s">
<label for="bulkPassword" data-i18n="bulkPassword">Mot de passe root</label>
<input id="bulkPassword" class="inp mono" type="password" autocomplete="new-password" required>
<span class="lbl" data-i18n="cTLS">TLS</span>
<select id="bulkTLS" class="field">
<option value="fp" data-i18n="tlsFp">Empreinte du certificat (enregistrée au premier test)</option>
<option value="sys" data-i18n="tlsSys">Autorités du système</option>
<option value="insecure" data-i18n="tlsInsecure">Sans vérification (labo uniquement)</option>
</select>
</div>
<ul id="bulkRes" class="bulk-res"></ul>
</div>
<footer>
<span class="sp"></span>
<button class="btn ghost" type="button" data-close data-i18n="close">Fermer</button>
<button id="bulkGo" class="btn primary" type="submit" data-i18n="bulkGo">Ajouter et tester</button>
</footer>
</form>
</dialog>
<!-- SSH access of a node (administrators) -->
<dialog id="nodeDlg" aria-labelledby="nodeTitle">
<form class="dlg" id="nodeForm" method="dialog">
<header>
<h2 id="nodeTitle"></h2>
<button class="icon-btn" type="button" data-close data-i18n-aria="close"><svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg></button>
</header>
<div class="dlg-body">
<p class="muted small" data-i18n="nodeHelp"></p>
<div class="field-grid">
<label for="nHost" data-i18n="nHost">Adresse SSH</label>
<input id="nHost" class="inp mono" autocomplete="off" spellcheck="false">
<label for="nPort" data-i18n="cSSHPort">Port SSH</label>
<input id="nPort" class="inp mono" type="number" min="1" max="65535" placeholder="22">
<label for="nPassword" data-i18n="nPassword">Mot de passe root</label>
<input id="nPassword" class="inp mono" type="password" autocomplete="new-password" spellcheck="false">
<span></span>
<label class="switch-row"><input id="nClear" type="checkbox" class="switch"><span data-i18n="nClear">Utiliser le mot de passe du cluster</span></label>
<span class="lbl" data-i18n="cSSHKey">Clé d'hôte SSH</span>
<span id="nKey" class="mono small muted"></span>
</div>
<p id="nErr" class="login-err" role="alert" hidden></p>
</div>
<footer>
<button id="nForget" class="btn ghost" type="button" data-i18n="forgetKey">Oublier la clé</button>
<span class="sp"></span>
<button class="btn ghost" type="button" data-close data-i18n="cancel">Annuler</button>
<button class="btn primary" type="submit" data-i18n="save">Enregistrer</button>
</footer>
</form>
</dialog>
<!-- Node replication profile (administrators) -->
<dialog id="profileDlg" class="wide" aria-labelledby="profileTitle">
<form class="dlg" id="profileForm" method="dialog">
<header>
<h2 id="profileTitle"></h2>
<button class="icon-btn" type="button" data-close data-i18n-aria="close"><svg viewBox="0 0 24 24"><path d="M6 6l12 12M18 6 6 18"/></svg></button>
</header>
<div class="dlg-body">
<div class="field-grid">
<label for="pName" data-i18n="cName">Nom</label>
<input id="pName" class="inp" required autocomplete="off" spellcheck="false">
<label for="pMaster" data-i18n="pMaster">Nœud maître</label>
<select id="pMaster" class="field" required></select>
</div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="pTargets">Nœuds cibles</h3></div>
<div id="pTargets" class="chip-field"></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="pItems">Éléments répliqués</h3></div>
<div id="pItems" class="chip-field"></div>
<p class="muted small mt-s" data-i18n="pItemsHelp"></p>
<label class="switch-row mt-s"><input id="pPrune" type="checkbox" class="switch"><span data-i18n="pPrune">Supprimer sur les cibles ce que le maître n'a pas</span></label>
<p class="muted small" data-i18n="pPruneHelp"></p>
<div class="sec-head mt"><h3 class="lbl" data-i18n="polTitle">Politiques par champ</h3></div>
<p class="muted small" data-i18n="pPolHelp"></p>
<div class="erow-head pol"><span data-i18n="colKind">Type</span><span data-i18n="colField">Champ</span><span data-i18n="colPolicy">Politique</span><span></span></div>
<div id="pPolRows" class="erows"></div>
<div class="toolbar"><button id="pPolAdd" class="btn tool" type="button" data-i18n="addRow">+ Ajouter</button></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="pProtect">Lignes protégées des fichiers</h3></div>
<p class="muted small" data-i18n="pProtectHelp"></p>
<div id="pProtect" class="erows"></div>
<div class="sec-head mt"><h3 class="lbl" data-i18n="pMapping">Mapping par cible</h3></div>
<p class="muted small" data-i18n="pMappingHelp"></p>
<div id="pMapRows" class="erows"></div>
<p id="pErr" class="login-err" role="alert" hidden></p>
</div>
<footer>
<button id="pDelete" class="btn danger" type="button" data-i18n="delete">Supprimer</button>
<span class="sp"></span>
<button class="btn ghost" type="button" data-close data-i18n="cancel">Annuler</button>
<button class="btn primary" type="submit" data-i18n="save">Enregistrer</button>
@@ -243,25 +418,15 @@
<div class="set-body">
<div class="set-side">
<nav id="setNav" class="set-nav" role="tablist" aria-orientation="vertical">
<button type="button" role="tab" data-tab="clusters">
<!-- Lucide "server" (ISC license) -->
<svg viewBox="0 0 24 24"><rect width="20" height="8" x="2" y="2" rx="2"/><rect width="20" height="8" x="2" y="14" rx="2"/><path d="M6 6h.01M6 18h.01"/></svg>
<span data-i18n="tabClusters">Clusters</span>
</button>
<button type="button" role="tab" data-tab="sync">
<svg viewBox="0 0 24 24"><path d="M3 12a9 9 0 0 1 15.7-6L21 8M21 3v5h-5M21 12a9 9 0 0 1-15.7 6L3 16M3 21v-5h5"/></svg>
<span data-i18n="tabSync">Synchronisation</span>
<span data-i18n="tabSync">Réplication</span>
</button>
<button type="button" role="tab" data-tab="users" data-admin>
<!-- Lucide "users" (ISC license) -->
<svg viewBox="0 0 24 24"><path d="M16 21v-2a4 4 0 0 0-4-4H6a4 4 0 0 0-4 4v2"/><circle cx="9" cy="7" r="4"/><path d="M22 21v-2a4 4 0 0 0-3-3.87M16 3.13a4 4 0 0 1 0 7.75"/></svg>
<span data-i18n="tabUsers">Utilisateurs</span>
</button>
<button type="button" role="tab" data-tab="config" data-admin>
<!-- Lucide "file-code" (ISC license) -->
<svg viewBox="0 0 24 24"><path d="M15 2H6a2 2 0 0 0-2 2v16a2 2 0 0 0 2 2h12a2 2 0 0 0 2-2V7Z"/><path d="M14 2v4a2 2 0 0 0 2 2h4M10 12l-2 2 2 2M14 12l2 2-2 2"/></svg>
<span data-i18n="tabConfig">Import / export</span>
</button>
<button type="button" role="tab" data-tab="account">
<!-- Lucide "key-round" (ISC license) -->
<svg viewBox="0 0 24 24"><path d="M2.586 17.414A2 2 0 0 0 2 18.828V21a1 1 0 0 0 1 1h3a1 1 0 0 0 1-1v-1a1 1 0 0 1 1-1h1a1 1 0 0 0 1-1v-1a1 1 0 0 1 1-1h.172a2 2 0 0 0 1.414-.586l.814-.814a6.5 6.5 0 1 0-4-4z"/><circle cx="16.5" cy="7.5" r=".5" fill="currentColor"/></svg>
@@ -283,30 +448,17 @@
</div>
<div class="set-panels">
<!-- Clusters -->
<div class="set-panel wide" role="tabpanel" data-panel="clusters">
<p class="ro-note" data-ro data-i18n="roNote" hidden></p>
<section class="set-section">
<div class="sec-head">
<h3 data-i18n="tabClusters">Clusters</h3>
<button id="addCluster" class="btn tool primary" type="button" data-admin-only data-i18n="addCluster">+ Ajouter un cluster</button>
</div>
<p class="muted small" data-i18n="clustersHelp"></p>
<div id="clusterList" class="clist"></div>
</section>
</div>
<!-- Synchronization settings -->
<div class="set-panel" role="tabpanel" data-panel="sync" hidden>
<div class="set-panel" role="tabpanel" data-panel="sync">
<p class="ro-note" data-ro data-i18n="roNote" hidden></p>
<section class="set-section" data-edit>
<h3 data-i18n="kindsTitle">Types synchronisés</h3>
<h3 data-i18n="kindsTitle">Éléments répliqués</h3>
<div id="kindChips" class="chip-field"></div>
<p class="muted small mt-s" data-i18n="kindsHelp"></p>
</section>
<section class="set-section danger" data-edit>
<h3 data-i18n="pruneTitle">Suppression</h3>
<label class="switch-row"><input id="pruneSw" type="checkbox" class="switch"><span data-i18n="pruneLabel">Supprimer chez les clients les objets absents de la référence</span></label>
<label class="switch-row"><input id="pruneSw" type="checkbox" class="switch"><span data-i18n="pruneLabel">Supprimer sur les cibles les objets absents du primaire</span></label>
<p class="muted small mt-s" data-i18n="pruneHelp"></p>
</section>
<section class="set-section span" data-edit>
@@ -336,31 +488,6 @@
</section>
</div>
<!-- Import / export -->
<div class="set-panel" role="tabpanel" data-panel="config" hidden>
<section class="set-section">
<h3 data-i18n="exportTitle">Exporter</h3>
<p class="muted small" data-i18n="exportHelp"></p>
<a id="exportBtn" class="btn" href="api/config/export" download="confsync.yaml">
<svg viewBox="0 0 24 24" aria-hidden="true"><path d="M12 3v12M7 10l5 5 5-5M4 17v3h16v-3"/></svg>
<span data-i18n="exportBtn">Télécharger confsync.yaml</span>
</a>
</section>
<section class="set-section">
<h3 data-i18n="importTitle">Importer</h3>
<p class="muted small" data-i18n="importHelp"></p>
<textarea id="importText" class="inp" spellcheck="false" placeholder="reference:&#10; name: ref&#10; …"></textarea>
<div class="toolbar">
<input id="importFile" type="file" accept=".yaml,.yml,text/yaml" hidden>
<button id="importPick" class="btn tool" type="button" data-i18n="importPick">Choisir un fichier…</button>
<label class="switch-row"><input id="importReplace" type="checkbox" class="switch"><span data-i18n="importReplace">Supprimer les clusters absents du fichier</span></label>
<span class="spacer"></span>
<button id="importBtn" class="btn primary" type="button" data-i18n="importBtn">Importer</button>
</div>
<div id="importRes" class="notes import-res"></div>
</section>
</div>
<!-- Account -->
<div class="set-panel" role="tabpanel" data-panel="account" hidden>
<section class="set-section">
web/login.html
+14 -1
View File
@@ -25,7 +25,20 @@
<svg viewBox="0 0 32 32" aria-hidden="true"><rect width="32" height="32" rx="8"/><path d="M8.5 14.5a8 8 0 0 1 13.6-4.6M22.6 6v4.2h-4.2M23.5 17.5a8 8 0 0 1-13.6 4.6M9.4 26v-4.2h4.2"/></svg>
<span>pve-confsync</span>
</div>
<p class="muted" data-i18n="intro">Synchronisation de configuration Proxmox VE</p>
<p class="muted" data-i18n="intro">Réplication de configuration Proxmox VE</p>
<form id="setupForm" class="setup-form" method="post" action="auth/setup" hidden>
<h2 data-i18n="setupTitle">Premier démarrage</h2>
<p data-i18n="setupIntro">La base est vide : créez le compte administrateur de pve-confsync.</p>
<label for="sUser" data-i18n="user">Utilisateur</label>
<input id="sUser" name="user" type="text" value="admin" autocomplete="username" autocapitalize="none" spellcheck="false" required>
<label for="sPass" data-i18n="pass">Mot de passe</label>
<input id="sPass" name="pass" type="password" autocomplete="new-password" minlength="12" required>
<label for="sPass2" data-i18n="pass2">Confirmation</label>
<input id="sPass2" name="pass2" type="password" autocomplete="new-password" minlength="12" required>
<p id="sErr" class="login-err" role="alert" hidden></p>
<button class="btn primary" type="submit" data-i18n="setupBtn">Créer le compte et continuer</button>
</form>
<a id="ssoBtn" class="btn primary" href="auth/oidc" hidden data-i18n="sso">Se connecter avec le SSO</a>
<div id="sep" class="login-sep" hidden data-i18n="or">ou</div>
web/login.js
+24 -6
View File
@@ -1,20 +1,26 @@
// Login page: language, theme, local form and/or OIDC button (from /auth/config).
(function () {
var I18N = {
fr: { intro: 'Synchronisation de configuration Proxmox VE', user: 'Utilisateur', pass: 'Mot de passe',
fr: { intro: 'Réplication de configuration Proxmox VE', user: 'Utilisateur', pass: 'Mot de passe',
submit: 'Se connecter', sso: 'Se connecter avec le SSO', or: 'ou',
error: 'Utilisateur ou mot de passe incorrect.', theme: 'Thème clair / sombre',
demo: 'Mode démo : comptes {a}, {o} et {l}, mot de passe {p}.' },
en: { intro: 'Proxmox VE configuration synchronization', user: 'User', pass: 'Password',
demo: 'Mode démo : comptes {a}, {o} et {l}, mot de passe {p}.',
setupTitle: 'Premier démarrage', setupIntro: 'La base est vide : créez le compte administrateur de pve-confsync (12 caractères minimum).',
pass2: 'Confirmation', setupBtn: 'Créer le compte et continuer',
mismatch: 'Les deux mots de passe diffèrent.', invalid: 'Nom ou mot de passe refusé (12 caractères minimum).' },
en: { intro: 'Proxmox VE configuration replication', user: 'User', pass: 'Password',
submit: 'Sign in', sso: 'Sign in with SSO', or: 'or',
error: 'Wrong user or password.', theme: 'Light / dark theme',
demo: 'Demo mode: accounts {a}, {o} and {l}, password {p}.' },
demo: 'Demo mode: accounts {a}, {o} and {l}, password {p}.',
setupTitle: 'First start', setupIntro: 'The database is empty: create the administrator account of pve-confsync (12 characters minimum).',
pass2: 'Confirm', setupBtn: 'Create the account and continue',
mismatch: 'The two passwords differ.', invalid: 'Name or password refused (12 characters minimum).' },
};
function get(k) { try { return localStorage.getItem('confsync.' + k); } catch (e) { return null; } }
function set(k, v) { try { localStorage.setItem('confsync.' + k, v); } catch (e) {} }
var lang = get('lang');
if (!I18N[lang]) lang = /^en\b/i.test(navigator.language || '') ? 'en' : 'fr';
var cfg = { local: true, oidc: false, demo: false };
var cfg = { local: true, oidc: false, demo: false, setup: false };
function el(id) { return document.getElementById(id); }
function code(s) { var c = document.createElement('code'); c.textContent = s; return c; }
@@ -25,6 +31,9 @@
document.querySelectorAll('[data-i18n-title]').forEach(function (e) { e.title = d[e.dataset.i18nTitle]; });
document.querySelectorAll('[data-i18n-aria]').forEach(function (e) { e.setAttribute('aria-label', d[e.dataset.i18nAria]); });
document.querySelectorAll('#langSwitch [data-lang]').forEach(function (b) { b.setAttribute('aria-checked', String(b.dataset.lang === lang)); });
var se = q.get('setup');
el('sErr').hidden = !se;
if (se) el('sErr').textContent = d[se === 'mismatch' ? 'mismatch' : 'invalid'];
var h = el('demoHint');
h.textContent = '';
if (cfg.demo) {
@@ -50,9 +59,18 @@
el('ret').value = ret;
el('err').hidden = q.get('e') !== '1';
el('ssoBtn').href = 'auth/oidc?r=' + encodeURIComponent(ret);
el('setupForm').addEventListener('submit', function (e) {
if (el('sPass').value !== el('sPass2').value) {
e.preventDefault();
el('sErr').textContent = I18N[lang].mismatch;
el('sErr').hidden = false;
}
});
function show() {
el('localForm').hidden = !cfg.local;
// empty database: the first administrator is created here
el('setupForm').hidden = !cfg.setup;
el('localForm').hidden = !cfg.local || cfg.setup;
el('ssoBtn').hidden = !cfg.oidc;
el('sep').hidden = !(cfg.local && cfg.oidc);
el('submitBtn').className = cfg.oidc ? 'btn' : 'btn primary';
web/style.css
+124 -6
View File
@@ -32,6 +32,7 @@
--k-zone: #1d6fd6;
--k-vnet: #1f9aa3;
--k-subnet: #5f8a1f;
--k-acc: #b45309;
--mono: ui-monospace, "SF Mono", "JetBrains Mono", "Cascadia Code", Menlo, Consolas, monospace;
--sans: system-ui, -apple-system, "Segoe UI", Roboto, "Helvetica Neue", Arial, sans-serif;
@@ -65,6 +66,7 @@
--k-zone: #5aa2f5;
--k-vnet: #3cc3cc;
--k-subnet: #9ccc54;
--k-acc: #f0a050;
color-scheme: dark;
}
@@ -94,6 +96,7 @@
--k-zone: #5aa2f5;
--k-vnet: #3cc3cc;
--k-subnet: #9ccc54;
--k-acc: #f0a050;
color-scheme: dark;
}
}
@@ -109,6 +112,9 @@ body {
padding-bottom: 44px;
}
button, input, select, textarea { font: inherit; color: inherit; }
/* native dropdown lists: follow the theme (light text on a light popup otherwise) */
select { color-scheme: inherit; }
option, optgroup { background-color: var(--panel); color: var(--text); }
button { cursor: pointer; }
svg { fill: none; stroke: currentColor; stroke-width: 2; stroke-linecap: round; stroke-linejoin: round; }
.muted { color: var(--muted); }
@@ -248,20 +254,21 @@ body.busy .progress::after {
position: relative; display: flex; flex-direction: column; gap: 4px; min-width: 0;
padding: 10px 12px 10px 14px; text-align: left;
background: var(--panel); border: 1px solid var(--border); border-radius: var(--radius);
border-left: 3px solid var(--st, var(--border));
}
button.ccard:hover { background: var(--row-hover); }
.ccard[aria-selected="true"] { border-color: var(--accent); border-left-color: var(--st, var(--accent)); box-shadow: 0 0 0 3px var(--accent-soft); }
.ccard.ref { --st: var(--accent); background: color-mix(in srgb, var(--accent) 5%, var(--panel)); cursor: default; }
.ccard.st-ok { --st: var(--ok); }
.ccard.st-diff { --st: var(--warn); }
.ccard.st-error { --st: var(--err); }
.ccard[aria-selected="true"] { border-color: var(--accent); box-shadow: 0 0 0 1px var(--accent); }
.ccard.ref { background: color-mix(in srgb, var(--accent) 5%, var(--panel)); cursor: default; }
.ccard .c-head { display: flex; align-items: center; gap: 8px; min-width: 0; }
.ccard .c-name { font-weight: 650; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.ccard .c-role {
flex: none; padding: 0 5px; border-radius: 5px; font-size: 10px; font-weight: 700; letter-spacing: .04em;
color: var(--accent); background: var(--accent-soft);
}
/* status badge, same shape as the role badge */
.ccard .c-role.st { --c: var(--muted); color: var(--c); background: color-mix(in srgb, var(--c) 14%, transparent); box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--c) 40%, transparent); }
.ccard .c-role.st-ok { --c: var(--ok); }
.ccard .c-role.st-diff { --c: var(--warn); }
.ccard .c-role.st-error { --c: var(--err); }
.ccard .c-ver { margin-left: auto; flex: none; font: 11.5px var(--mono); color: var(--muted); }
.ccard .c-url { font: 11.5px var(--mono); color: var(--muted); overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
.ccard .c-counts { display: flex; flex-wrap: wrap; gap: 4px; min-height: 20px; align-items: center; }
@@ -362,6 +369,7 @@ button.ccard:hover { background: var(--row-hover); }
.kind.sdn-zone { --k: var(--k-zone); }
.kind.sdn-vnet { --k: var(--k-vnet); }
.kind.sdn-subnet { --k: var(--k-subnet); }
.kind.pve-group, .kind.pve-role, .kind.pve-user, .kind.pve-password { --k: var(--k-acc); }
/* status / mode badges */
.badge {
@@ -688,3 +696,113 @@ body.login-page { min-height: 100vh; padding: 16px; display: grid; place-items:
.mt-s { margin-top: 8px !important; }
.erow .acts { display: flex; gap: 2px; }
.import-res { margin: 10px 0 0; }
/* ---------- drop-down menu of the filter bar (targets) ---------- */
.menu { position: relative; }
.menu > summary {
list-style: none; display: inline-flex; align-items: center; gap: 6px; height: 32px; padding: 0 10px;
border: 1px solid var(--border); border-radius: 8px; background: var(--panel); font-size: 13px; font-weight: 550; cursor: pointer; white-space: nowrap;
}
.menu > summary::-webkit-details-marker { display: none; }
.menu > summary svg { width: 15px; height: 15px; }
.menu > summary .n { font: 600 11px/1 var(--mono); padding: 3px 6px; border-radius: 10px; background: var(--accent-soft); color: var(--accent); }
.menu[open] > summary { border-color: var(--accent); box-shadow: 0 0 0 3px var(--accent-soft); }
.menu .pop {
position: absolute; z-index: 30; top: calc(100% + 6px); left: 0; min-width: 18rem; max-height: 70vh; overflow-y: auto;
padding: 8px; border: 1px solid var(--border); border-radius: 10px; background: var(--panel); box-shadow: var(--shadow);
}
.menu.right .pop { left: auto; right: 0; }
.menu .grp { margin: 6px 4px 2px; font-size: 10.5px; font-weight: 650; letter-spacing: .04em; text-transform: uppercase; color: var(--muted); }
.menu .grp:first-child { margin-top: 0; }
.menu label.opt { display: flex; align-items: center; gap: 8px; padding: 5px 6px; border-radius: 6px; font-size: 13px; cursor: pointer; }
.menu label.opt:hover { background: var(--row-hover); }
.menu label.opt input { accent-color: var(--accent); }
.menu label.opt .h { margin-left: auto; font-size: 11.5px; color: var(--muted); }
.menu .pop .foot { display: flex; gap: 6px; margin-top: 6px; padding-top: 6px; border-top: 1px solid var(--border); }
.menu .pop .foot .note-s { font-size: 11.5px; color: var(--muted); }
/* ---------- clusters view ---------- */
.cl-view { padding: 12px 20px 20px; }
.cl-tools { display: flex; flex-wrap: wrap; gap: 8px; align-items: center; margin-bottom: 10px; }
.cl-tools p { margin: 0; flex: 1 1 320px; }
.citem.cl { grid-template-columns: minmax(9rem, max-content) minmax(0, 1fr) max-content; }
.citem.cl.primary { border-left-color: var(--accent); background: color-mix(in srgb, var(--accent) 4%, var(--panel)); }
.citem.cl.disabled { opacity: .7; }
.citem .meta { grid-column: 1 / -1; display: flex; flex-wrap: wrap; gap: 4px 14px; font-size: 12px; color: var(--muted); }
.citem .meta span { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; max-width: 100%; }
.citem .meta b { font-weight: 600; color: var(--text); }
.citem .meta .mono { font-size: 11.5px; }
.rbadge { font: 700 10px/1.5 var(--mono); padding: 0 6px; border-radius: 4px; text-transform: uppercase; letter-spacing: .03em;
color: var(--c); background: color-mix(in srgb, var(--c) 14%, transparent); box-shadow: inset 0 0 0 1px color-mix(in srgb, var(--c) 40%, transparent); }
.rbadge.primary { --c: var(--accent); }
.rbadge.target { --c: var(--ok); }
.rbadge.disabled { --c: var(--op-same); }
.citem select.field { height: 30px; width: auto; }
.bulk-res { list-style: none; margin: 10px 0 0; padding: 0; font: 12px var(--mono); }
.bulk-res li { padding: 3px 0; border-bottom: 1px dashed var(--border); }
.bulk-res li.ok { color: var(--ok); }
.bulk-res li.ko { color: var(--err); }
.setup-form { width: 100%; display: flex; flex-direction: column; gap: 6px; }
.setup-form h2 { margin: 0 0 4px; font-size: 15px; text-align: center; }
.setup-form p { margin: 0 0 6px; font-size: 12.5px; color: var(--muted); }
/* dry-run: replicated items (replicate / do not replicate) */
.plan-kinds { display: grid; gap: 6px; padding: 12px 20px; border-bottom: 1px solid var(--border); background: var(--panel-2); }
.plan-kinds .pk-head { display: flex; flex-wrap: wrap; align-items: baseline; gap: 4px 10px; margin-bottom: 2px; }
.plan-kinds .pk-row { display: grid; grid-template-columns: 6.5rem minmax(0, 1fr) 2.5rem auto; align-items: center; gap: 10px; }
.plan-kinds .pk-row.off .pk-name, .plan-kinds .pk-row.off .kind { opacity: .55; }
.plan-kinds .pk-n { text-align: right; font-variant-numeric: tabular-nums; color: var(--muted); }
.plan-kinds .seg button { padding: 3px 10px; font-size: 12px; }
.plan-kinds .seg button:disabled { cursor: default; }
.plan-kinds .seg button:disabled:not([aria-checked="true"]) { opacity: .55; }
@media (max-width: 560px) { .plan-kinds .pk-row { grid-template-columns: minmax(0, 1fr) auto; } .plan-kinds .pk-row .kind, .plan-kinds .pk-n { display: none; } }
/* ---------- node replication ---------- */
/* the two replications, side by side in the top bar */
.mode-sw { flex: none; }
.mode-sw button { display: inline-flex; align-items: center; gap: 6px; height: 28px; padding: 0 12px; font-weight: 600; }
.mode-sw svg { width: 15px; height: 15px; }
.mode-sw button[aria-checked="true"] { background: var(--accent); color: #fff; }
body.mode-node .brand svg rect { fill: var(--k-vnet); }
.kind.network { --k: var(--k-zone); }
.kind.dns { --k: var(--k-vnet); }
.kind.multipath, .kind.wwids { --k: var(--k-acc); }
.kind.chrony { --k: var(--k-dc); }
.kind.sysctl { --k: var(--k-subnet); }
.kind.interfaces-d { --k: var(--k-zone); }
.badge.network { --c: var(--k-zone); margin-left: 4px; }
/* unified diff of a file */
.udiff { margin: 4px 0 0; padding: 6px 0; border: 1px solid var(--border); border-radius: 8px; background: var(--panel-2);
font: 12px/1.45 var(--mono); overflow-x: auto; max-height: 420px; overflow-y: auto; }
.udiff div { padding: 0 10px; white-space: pre; min-height: 1.45em; }
.udiff .d-add { color: var(--op-create); background: color-mix(in srgb, var(--op-create) 10%, transparent); }
.udiff .d-del { color: var(--op-delete); background: color-mix(in srgb, var(--op-delete) 10%, transparent); }
.udiff .d-hunk { color: var(--info); padding-top: 4px; }
.plan .udiff { margin-left: 1.2rem; }
.plan .reloads { margin: 6px 0 0; color: var(--muted); font-size: 12px; }
.sec-title { margin: 18px 0 8px; font-size: 11px; font-weight: 650; letter-spacing: .04em; text-transform: uppercase; color: var(--muted); }
.sec-title:first-of-type { margin-top: 6px; }
.citem.node { grid-template-columns: minmax(9rem, max-content) minmax(0, 1fr) max-content; }
.citem.prof { grid-template-columns: minmax(9rem, max-content) minmax(0, 1fr) max-content; }
.citem .items { grid-column: 1 / -1; display: flex; flex-wrap: wrap; gap: 4px; }
.citem .items .kind { width: auto; }
dialog.wide { width: min(980px, calc(100vw - 32px)); }
/* mapping of one target in the profile editor */
.pmap { display: grid; grid-template-columns: 11rem minmax(0, 1fr); gap: 6px 10px; align-items: start; padding: 8px 10px;
border: 1px solid var(--border); border-radius: 8px; }
.pmap > b { grid-column: 1 / -1; font: 600 12.5px var(--mono); }
.pmap label { font-size: 12px; color: var(--muted); padding-top: 6px; }
.pmap textarea { min-height: 3.2em; font-family: var(--mono); font-size: 12px; }
.pprot { display: grid; grid-template-columns: 11rem minmax(0, 1fr); gap: 6px 10px; align-items: start; }
.pprot textarea { min-height: 2.4em; font-family: var(--mono); font-size: 12px; }
@media (max-width: 1100px) {
.mode-sw span { display: none; }
.mode-sw button { padding: 0 9px; }
}
@media (max-width: 760px) {
.pmap, .pprot { grid-template-columns: 1fr; }
}