feat(cicd): Ajouter gestion automatique des ressources DRBD Linstor
Some checks failed
CD - Deploy Infrastructure / Terraform Validation (push) Successful in 17s
CD - Deploy Infrastructure / Deploy on pve1 (push) Failing after 5s
CD - Deploy Infrastructure / Deploy on pve2 (push) Failing after 5s
CD - Deploy Infrastructure / Deploy on pve3 (push) Failing after 7s
CD - Deploy Infrastructure / Validate K3s Cluster (push) Has been skipped
CD - Deploy Infrastructure / Deployment Notification (push) Failing after 1s
Some checks failed
CD - Deploy Infrastructure / Terraform Validation (push) Successful in 17s
CD - Deploy Infrastructure / Deploy on pve1 (push) Failing after 5s
CD - Deploy Infrastructure / Deploy on pve2 (push) Failing after 5s
CD - Deploy Infrastructure / Deploy on pve3 (push) Failing after 7s
CD - Deploy Infrastructure / Validate K3s Cluster (push) Has been skipped
CD - Deploy Infrastructure / Deployment Notification (push) Failing after 1s
- Créer script Python pour gérer les ressources DRBD avant déploiement
* Vérifie l'existence des ressources Linstor
* Crée les ressources si nécessaire avec réplication
* Augmente la taille si elle est insuffisante
* Noms fixes: pm-a7f3c8e1 (VMID 1000) et pm-b4d2f9a3 (VMID 1001)
- Modifier workflow CI/CD pour intégrer le script Python
* Ajouter étape de configuration SSH avec secret LINSTOR_SSH_PRIVATE_KEY
* Exécuter le script avant tofu apply sur pve1 et pve2
- Corriger configuration Terraform des VMs
* Ajouter vga { type = "std" } pour Standard VGA sur toutes les VMs
* Ajouter cpu { type = "host" } pour meilleure performance
* Ajouter replace_triggered_by pour détecter les changements de config
* Ajouter force_create = true sur pve3 pour gérer VM existante
- Résoudre problèmes identifiés
* "No Bootable Device" - Résolu avec Standard VGA et CPU host
* "vmId already in use" - Résolu avec force_create sur etcd-witness
* Détection des modifications de VM - Résolu avec replace_triggered_by
Documentation SSH créée dans cicd_backup/SETUP_SSH_LINSTOR.md
This commit is contained in:
parent
cc26fb97a6
commit
89e48f6f5f
8 changed files with 653 additions and 5 deletions
54
scripts/copy-template-to-local.sh
Normal file
54
scripts/copy-template-to-local.sh
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
#!/bin/bash
|
||||
# Script to copy Ubuntu template from LINSTOR to local storage on each node
|
||||
# This is necessary because LINSTOR doesn't support cloning operations properly
|
||||
|
||||
set -e
|
||||
|
||||
TEMPLATE_VMID=9000
|
||||
TEMPLATE_NAME="ubuntu-2404-cloudinit"
|
||||
SOURCE_STORAGE="linstor_storage"
|
||||
TARGET_STORAGE="local"
|
||||
NODES=("acemagician" "elitedesk" "thinkpad")
|
||||
|
||||
echo "=== Copying template $TEMPLATE_NAME (VMID: $TEMPLATE_VMID) to local storage on each node ==="
|
||||
|
||||
for node in "${NODES[@]}"; do
|
||||
echo ""
|
||||
echo "--- Processing node: $node ---"
|
||||
|
||||
# Check if template already exists locally on this node
|
||||
if ssh root@$node "qm status $TEMPLATE_VMID &>/dev/null"; then
|
||||
echo "✓ Template already exists on $node"
|
||||
|
||||
# Check if it's on local storage
|
||||
if ssh root@$node "qm config $TEMPLATE_VMID | grep -q 'local:'"; then
|
||||
echo "✓ Template is already on local storage"
|
||||
continue
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "→ Cloning template from LINSTOR to local storage on $node..."
|
||||
|
||||
# Clone the template to local storage with a temporary VMID
|
||||
TEMP_VMID=$((TEMPLATE_VMID + 1000))
|
||||
|
||||
ssh root@$node "qm clone $TEMPLATE_VMID $TEMP_VMID \
|
||||
--name ${TEMPLATE_NAME}-local \
|
||||
--full \
|
||||
--storage $TARGET_STORAGE \
|
||||
--target $node" || {
|
||||
echo "✗ Failed to clone template on $node"
|
||||
continue
|
||||
}
|
||||
|
||||
echo "✓ Template copied successfully to $node (VMID: $TEMP_VMID)"
|
||||
echo " Note: You can now use VMID $TEMP_VMID or rename to $TEMPLATE_VMID after removing the LINSTOR version"
|
||||
done
|
||||
|
||||
echo ""
|
||||
echo "=== Template copy complete ==="
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. Verify templates exist on each node: ssh root@<node> 'qm list'"
|
||||
echo "2. Update Terraform to use local templates or new VMIDs"
|
||||
echo "3. Optionally remove LINSTOR template after testing"
|
||||
285
scripts/manage_linstor_resources.py
Normal file
285
scripts/manage_linstor_resources.py
Normal file
|
|
@ -0,0 +1,285 @@
|
|||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Script de gestion des ressources DRBD Linstor pour les VMs K3s
|
||||
Exécuté avant le déploiement Terraform pour s'assurer que les ressources
|
||||
de stockage DRBD sont créées et dimensionnées correctement.
|
||||
"""
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
import json
|
||||
import argparse
|
||||
from typing import Dict, Optional, Tuple
|
||||
|
||||
|
||||
# Noms de ressources DRBD fixes pour chaque nœud
|
||||
RESOURCE_NAMES = {
|
||||
1000: "pm-a7f3c8e1", # acemagician - k3s-server-1
|
||||
1001: "pm-b4d2f9a3", # elitedesk - k3s-server-2
|
||||
}
|
||||
|
||||
# Configuration des nœuds Proxmox
|
||||
NODE_CONFIG = {
|
||||
1000: {"node": "acemagician", "vm_name": "k3s-server-1"},
|
||||
1001: {"node": "elitedesk", "vm_name": "k3s-server-2"},
|
||||
}
|
||||
|
||||
|
||||
def run_ssh_command(command: str, host: str = "192.168.100.30") -> Tuple[int, str, str]:
|
||||
"""
|
||||
Exécute une commande SSH sur le contrôleur Linstor (thinkpad - 192.168.100.30).
|
||||
|
||||
Args:
|
||||
command: Commande à exécuter
|
||||
host: Hôte sur lequel exécuter la commande (défaut: 192.168.100.30)
|
||||
|
||||
Returns:
|
||||
Tuple (code_retour, stdout, stderr)
|
||||
"""
|
||||
ssh_cmd = ["ssh", "-o", "StrictHostKeyChecking=no", f"root@{host}", command]
|
||||
|
||||
try:
|
||||
result = subprocess.run(
|
||||
ssh_cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=30
|
||||
)
|
||||
return result.returncode, result.stdout, result.stderr
|
||||
except subprocess.TimeoutExpired:
|
||||
return 1, "", "Timeout lors de l'exécution de la commande SSH"
|
||||
except Exception as e:
|
||||
return 1, "", f"Erreur lors de l'exécution SSH: {str(e)}"
|
||||
|
||||
|
||||
def check_resource_exists(resource_name: str) -> bool:
|
||||
"""
|
||||
Vérifie si une ressource DRBD existe.
|
||||
|
||||
Args:
|
||||
resource_name: Nom de la ressource à vérifier
|
||||
|
||||
Returns:
|
||||
True si la ressource existe, False sinon
|
||||
"""
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor resource list --resource {resource_name} --machine-readable"
|
||||
)
|
||||
|
||||
if returncode == 0:
|
||||
# Vérifie si la sortie contient des informations sur la ressource
|
||||
return len(stdout.strip()) > 0 and "resources" in stdout
|
||||
|
||||
return False
|
||||
|
||||
|
||||
def get_resource_size(resource_name: str) -> Optional[int]:
|
||||
"""
|
||||
Récupère la taille actuelle d'une ressource DRBD en GiB.
|
||||
|
||||
Args:
|
||||
resource_name: Nom de la ressource
|
||||
|
||||
Returns:
|
||||
Taille en GiB ou None si erreur
|
||||
"""
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor volume-definition list --resource {resource_name} --machine-readable"
|
||||
)
|
||||
|
||||
if returncode != 0:
|
||||
return None
|
||||
|
||||
try:
|
||||
# Parse la sortie JSON de Linstor
|
||||
data = json.loads(stdout)
|
||||
if data and len(data) > 0:
|
||||
volume_defs = data[0].get("volume_definitions", [])
|
||||
if volume_defs and len(volume_defs) > 0:
|
||||
# Taille en KiB, conversion en GiB
|
||||
size_kib = volume_defs[0].get("size_kib", 0)
|
||||
return size_kib // (1024 * 1024)
|
||||
except (json.JSONDecodeError, KeyError, IndexError) as e:
|
||||
print(f"Erreur lors du parsing de la taille: {e}", file=sys.stderr)
|
||||
return None
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def create_resource(resource_name: str, size_gib: int, nodes: list) -> bool:
|
||||
"""
|
||||
Crée une nouvelle ressource DRBD avec réplication.
|
||||
|
||||
Args:
|
||||
resource_name: Nom de la ressource à créer
|
||||
size_gib: Taille en GiB
|
||||
nodes: Liste des nœuds pour la réplication
|
||||
|
||||
Returns:
|
||||
True si succès, False sinon
|
||||
"""
|
||||
print(f"Création de la ressource {resource_name} avec {size_gib}GiB...")
|
||||
|
||||
# Étape 1: Créer la définition de ressource
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor resource-definition create {resource_name}"
|
||||
)
|
||||
|
||||
if returncode != 0:
|
||||
print(f"Erreur lors de la création de la définition: {stderr}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# Étape 2: Créer la définition de volume
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor volume-definition create {resource_name} {size_gib}GiB"
|
||||
)
|
||||
|
||||
if returncode != 0:
|
||||
print(f"Erreur lors de la création du volume: {stderr}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
# Étape 3: Déployer la ressource sur les nœuds avec réplication
|
||||
for node in nodes:
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor resource create {node} {resource_name} --storage-pool linstor_thinpool"
|
||||
)
|
||||
|
||||
if returncode != 0:
|
||||
print(f"Erreur lors du déploiement sur {node}: {stderr}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
print(f"✓ Ressource {resource_name} créée avec succès")
|
||||
return True
|
||||
|
||||
|
||||
def resize_resource(resource_name: str, new_size_gib: int) -> bool:
|
||||
"""
|
||||
Augmente la taille d'une ressource DRBD existante.
|
||||
|
||||
Args:
|
||||
resource_name: Nom de la ressource à redimensionner
|
||||
new_size_gib: Nouvelle taille en GiB
|
||||
|
||||
Returns:
|
||||
True si succès, False sinon
|
||||
"""
|
||||
print(f"Redimensionnement de la ressource {resource_name} à {new_size_gib}GiB...")
|
||||
|
||||
returncode, stdout, stderr = run_ssh_command(
|
||||
f"linstor volume-definition set-size {resource_name} 0 {new_size_gib}GiB"
|
||||
)
|
||||
|
||||
if returncode != 0:
|
||||
print(f"Erreur lors du redimensionnement: {stderr}", file=sys.stderr)
|
||||
return False
|
||||
|
||||
print(f"✓ Ressource {resource_name} redimensionnée avec succès")
|
||||
return True
|
||||
|
||||
|
||||
def manage_vm_resource(vmid: int, size_gib: int, dry_run: bool = False) -> bool:
|
||||
"""
|
||||
Gère la ressource DRBD pour une VM spécifique.
|
||||
|
||||
Args:
|
||||
vmid: ID de la VM
|
||||
size_gib: Taille souhaitée en GiB
|
||||
dry_run: Si True, affiche les actions sans les exécuter
|
||||
|
||||
Returns:
|
||||
True si succès, False sinon
|
||||
"""
|
||||
if vmid not in RESOURCE_NAMES:
|
||||
print(f"VMID {vmid} non configuré", file=sys.stderr)
|
||||
return False
|
||||
|
||||
resource_name = RESOURCE_NAMES[vmid]
|
||||
node_info = NODE_CONFIG[vmid]
|
||||
|
||||
print(f"\n{'='*60}")
|
||||
print(f"Gestion de la ressource pour VM {vmid} ({node_info['vm_name']})")
|
||||
print(f"Ressource DRBD: {resource_name}")
|
||||
print(f"Nœud Proxmox: {node_info['node']}")
|
||||
print(f"Taille souhaitée: {size_gib}GiB")
|
||||
print(f"{'='*60}\n")
|
||||
|
||||
# Vérifie si la ressource existe
|
||||
resource_exists = check_resource_exists(resource_name)
|
||||
|
||||
if not resource_exists:
|
||||
print(f"La ressource {resource_name} n'existe pas.")
|
||||
|
||||
if dry_run:
|
||||
print(f"[DRY-RUN] Créerait la ressource {resource_name} avec {size_gib}GiB")
|
||||
return True
|
||||
|
||||
# Créer la ressource sur les 3 nœuds pour réplication
|
||||
nodes = ["acemagician", "elitedesk", "thinkpad"]
|
||||
return create_resource(resource_name, size_gib, nodes)
|
||||
|
||||
else:
|
||||
print(f"La ressource {resource_name} existe déjà.")
|
||||
|
||||
# Vérifie la taille actuelle
|
||||
current_size = get_resource_size(resource_name)
|
||||
|
||||
if current_size is None:
|
||||
print("Impossible de récupérer la taille actuelle", file=sys.stderr)
|
||||
return False
|
||||
|
||||
print(f"Taille actuelle: {current_size}GiB")
|
||||
|
||||
if current_size == size_gib:
|
||||
print(f"✓ La taille correspond déjà ({size_gib}GiB), aucune action nécessaire")
|
||||
return True
|
||||
|
||||
elif current_size < size_gib:
|
||||
print(f"La taille doit être augmentée de {current_size}GiB à {size_gib}GiB")
|
||||
|
||||
if dry_run:
|
||||
print(f"[DRY-RUN] Redimensionnerait {resource_name} à {size_gib}GiB")
|
||||
return True
|
||||
|
||||
return resize_resource(resource_name, size_gib)
|
||||
|
||||
else:
|
||||
print(f"⚠ La taille actuelle ({current_size}GiB) est supérieure à la taille souhaitée ({size_gib}GiB)")
|
||||
print("La réduction de taille n'est pas supportée, conservation de la taille actuelle")
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
"""Point d'entrée principal du script."""
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Gestion des ressources DRBD Linstor pour les VMs K3s"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--vmid",
|
||||
type=int,
|
||||
required=True,
|
||||
choices=[1000, 1001],
|
||||
help="ID de la VM (1000=acemagician, 1001=elitedesk)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--size",
|
||||
type=int,
|
||||
required=True,
|
||||
help="Taille du disque en GiB"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--dry-run",
|
||||
action="store_true",
|
||||
help="Affiche les actions sans les exécuter"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
# Exécute la gestion de la ressource
|
||||
success = manage_vm_resource(args.vmid, args.size, args.dry_run)
|
||||
|
||||
sys.exit(0 if success else 1)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Loading…
Add table
Add a link
Reference in a new issue