sync-modules: mirrored modules of release 3.3.0-F-feature-simpl-33565.765.f7724799
+++ hpc/README.md +++ hpc/ovh/cloudinit.tf +++ hpc/ovh/credentials.tf +++ hpc/ovh/main.tf +++ hpc/ovh/outputs.tf +++ hpc/ovh/variables.tf +++ hpc/ovh/versions.tf
This commit is contained in:
parent
ade8a7358e
commit
53339e50a8
70
hpc/README.md
Normal file
70
hpc/README.md
Normal file
@ -0,0 +1,70 @@
|
||||
# hpc module
|
||||
|
||||
Provisions an HPC cluster and generates the consumer access credentials. One submodule per cloud provider, selected by the provisioner from the cloud provider of the provisioning request:
|
||||
|
||||
- `ovh`: one head node (Slurm controller, munge authority, NFS server, consumer entry point) plus N compute nodes (Slurm execution daemon, OpenMPI, NFS client) on a dedicated private network (vRack). All nodes are private only: internet egress goes through a managed cloud gateway attached to the cluster subnet, and the head node is reached from outside through a floating ip routed by the same gateway. The cluster bootstraps entirely from per-role cloud-init at first boot (Slurm configless, per-deployment munge key, shared `/home` over NFS, fixed uids cluster-wide) with no manual step.
|
||||
|
||||
## Contract
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `deployment_id` | Unique deployment identifier, used in resource names |
|
||||
| `cloud_account` | Cloud account (environment) name, second segment of the Vault credentials path, filled by the provisioner from the cloudAccount header via the `{CLOUD_ACCOUNT}` placeholder |
|
||||
| `requester_unique_id` | Requester unique id, third segment of the Vault credentials path, filled by the provisioner from the requesterUniqueId header via the `{REQUESTER_UNIQUE_ID}` placeholder |
|
||||
| `username` | Consumer username on the head node, defaults to `simpl_user`, last segment of the Vault credentials path. The user exists on every node with the same uid (Slurm runs jobs under it) but can only log in on the head node |
|
||||
| `admin_username` | Management username created on all nodes with SSH key access only, defaults to `simpladmin` |
|
||||
| `credentials_version` | Opaque rotation marker for the consumer credential, defaults to `1`. Changing the value regenerates the consumer password, applies it on the head node over SSH and republishes the fresh credential to Vault in the same apply, without touching any instance. Managed by the credential renewal workflow, not by the deployment script author |
|
||||
| `name_prefix` | Prefix for resource names, defaults to `simpl` |
|
||||
| `region` | OVH region for the instances and the private network, for example SBG5 |
|
||||
| `project_id` | Optional override of the OVH project; by default the provider resolves it from the `OVH_CLOUD_PROJECT_SERVICE` variable of the runner environment, which the provisioner derives from the `OVH_PROJECT_ID` key of the cloud account configuration |
|
||||
| `compute_node_count` | Number of compute nodes, defaults to 2 |
|
||||
| `head_flavor`, `compute_flavor` | Instance flavors per role, default `b3-8` |
|
||||
| `compute_cpus` | Schedulable CPUs per compute node declared to the Slurm controller, must not exceed the vCPUs of `compute_flavor`, defaults to 2 |
|
||||
| `image` | Cloud-init enabled image for every node, defaults to `Ubuntu 24.04` |
|
||||
| `cluster_cidr` | Private cluster network CIDR, defaults to `192.168.100.0/24`. The head node takes host 50, compute nodes host 60 onward; lower pool addresses stay free for the DHCP agent ports (an instance on a DHCP-port address is silently canceled by the cloud) |
|
||||
| `vlan_id` | vRack VLAN id for the private network, must be unused in the project, defaults to 212 |
|
||||
| `gateway_model` | Size of the cloud gateway providing the cluster egress, defaults to `s` |
|
||||
|
||||
Cloud credentials are never module inputs, they reach the provider from the runner environment.
|
||||
|
||||
Credentials, following the existing Vault contract: the module generates the consumer password and the management SSH keypair and writes them to Vault, `secret/{provider}/{cloud_account}/{requester_unique_id}/{username}` (keys username, password, host, port) and the `.../admin` sibling (keys username, ssh_private_key, ssh_public_key). The consumer password never travels through user data: it is applied on the head node over SSH, keyed on both the rotation marker and the head node instance id, so a recreated head node receives the credential again. Day 2 credential renewal follows the same Terraform rotation flow as the other catalog modules, changing the `credentials_version` input. No secret material is ever part of an output or of the user body. The Vault provider configuration is generated by the provisioner next to the deployment script, and it authenticates through the Vault Kubernetes auth method with the service account of the Terraform runner, against a Vault role configured at installation time.
|
||||
|
||||
Outputs, deliberately restricted to the entry point (the response payload returned to the backend carries strictly the head node access information, never the compute node details):
|
||||
|
||||
| Output | Description |
|
||||
|---|---|
|
||||
| `head_node_ip` | Public (floating) IPv4 address of the head node, the single entry point of the cluster |
|
||||
| `head_node_hostname` | Hostname of the head node |
|
||||
| `ssh_port` | SSH port of the head node |
|
||||
| `vmIps` | Entry point address list, kept for compatibility with the single VM notification flow |
|
||||
| `credentials_ref` | Vault path of the consumer credentials entry |
|
||||
|
||||
## Usage
|
||||
|
||||
The provisioner rewrites the `{HPC_SOURCE}` placeholder to this module's location for the cloud provider of the request (each catalog module has a placeholder derived from its directory name), so a deployment script only declares:
|
||||
|
||||
```hcl
|
||||
module "hpc" {
|
||||
source = "{HPC_SOURCE}"
|
||||
deployment_id = "{UUID}"
|
||||
region = "<region>"
|
||||
cloud_account = "{CLOUD_ACCOUNT}"
|
||||
requester_unique_id = "{REQUESTER_UNIQUE_ID}"
|
||||
|
||||
compute_node_count = 2
|
||||
}
|
||||
|
||||
output "head_node_ip" {
|
||||
value = module.hpc.head_node_ip
|
||||
}
|
||||
|
||||
output "vmIps" {
|
||||
value = module.hpc.vmIps
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
value = module.hpc.credentials_ref
|
||||
}
|
||||
```
|
||||
|
||||
The root file must re-export the module outputs it wants published to the deployment outputs.
|
||||
245
hpc/ovh/cloudinit.tf
Normal file
245
hpc/ovh/cloudinit.tf
Normal file
@ -0,0 +1,245 @@
|
||||
// Node bootstrap. Everything local to a node is applied by cloud-init at
|
||||
// first boot; the cluster forms without any manual step. Slurm runs in
|
||||
// configless mode: only the head node carries slurm.conf, compute nodes
|
||||
// fetch it from the controller. The munge key is generated per deployment
|
||||
// and delivered to every node through its user data.
|
||||
|
||||
locals {
|
||||
munge_key_b64 = random_bytes.munge_key.base64
|
||||
|
||||
// Written early by cloud-init to a staging path (the munge user does not
|
||||
// exist yet); runcmd installs it with the right owner after the packages.
|
||||
munge_stage_path = "/run/simpl/munge.key"
|
||||
|
||||
cluster_hosts = concat(
|
||||
["${local.head_ip} ${local.head_hostname}"],
|
||||
[for i in range(var.compute_node_count) : "${local.compute_ips[i]} ${local.compute_hostnames[i]}"],
|
||||
)
|
||||
|
||||
slurm_conf = join("\n", concat([
|
||||
"ClusterName=${lower(var.name_prefix)}-hpc",
|
||||
"SlurmctldHost=${local.head_hostname}(${local.head_ip})",
|
||||
"SlurmctldParameters=enable_configless",
|
||||
"AuthType=auth/munge",
|
||||
"SlurmUser=slurm",
|
||||
"CredType=cred/munge",
|
||||
"SelectType=select/cons_tres",
|
||||
"SelectTypeParameters=CR_Core",
|
||||
"ProctrackType=proctrack/linuxproc",
|
||||
"TaskPlugin=task/none",
|
||||
"ReturnToService=2",
|
||||
"SlurmctldPort=6817",
|
||||
"SlurmdPort=6818",
|
||||
"SrunPortRange=60001-60128",
|
||||
"StateSaveLocation=/var/lib/slurm/slurmctld",
|
||||
"SlurmdSpoolDir=/var/lib/slurm/slurmd",
|
||||
"SlurmctldLogFile=/var/log/slurm/slurmctld.log",
|
||||
"SlurmdLogFile=/var/log/slurm/slurmd.log",
|
||||
"MailProg=/bin/true",
|
||||
], [
|
||||
for i in range(var.compute_node_count) :
|
||||
"NodeName=${local.compute_hostnames[i]} NodeAddr=${local.compute_ips[i]} CPUs=${var.compute_cpus} State=UNKNOWN"
|
||||
], [
|
||||
"PartitionName=main Nodes=ALL Default=YES MaxTime=INFINITE State=UP",
|
||||
]))
|
||||
|
||||
hosts_runcmd = [for entry in local.cluster_hosts : "grep -qxF '${entry}' /etc/hosts || echo '${entry}' >> /etc/hosts"]
|
||||
|
||||
munge_runcmd = [
|
||||
"install -o munge -g munge -m 0400 ${local.munge_stage_path} /etc/munge/munge.key",
|
||||
"rm -f ${local.munge_stage_path}",
|
||||
"systemctl restart munge",
|
||||
]
|
||||
|
||||
// Every cluster node is on the private subnet only: the default route
|
||||
// comes from DHCP (the subnet gateway) but no name servers are provided,
|
||||
// so resolution is pinned to public resolvers. Runs on every boot and
|
||||
// before package installation; the last entry blocks until egress through
|
||||
// the gateway answers, capped at ~4 minutes.
|
||||
egress_bootcmd = [
|
||||
"mkdir -p /etc/systemd/resolved.conf.d",
|
||||
"printf '[Resolve]\\nDNS=1.1.1.1 8.8.8.8\\n' > /etc/systemd/resolved.conf.d/90-cluster.conf",
|
||||
"systemctl try-restart systemd-resolved || true",
|
||||
"for i in $(seq 1 48); do timeout 3 bash -c '</dev/tcp/archive.ubuntu.com/80' 2>/dev/null && break; sleep 5; done",
|
||||
]
|
||||
|
||||
head_cloud_config = {
|
||||
hostname = local.head_hostname
|
||||
prefer_fqdn_over_hostname = false
|
||||
package_update = true
|
||||
packages = [
|
||||
"slurmctld",
|
||||
"slurm-client",
|
||||
"munge",
|
||||
"nfs-kernel-server",
|
||||
"openmpi-bin",
|
||||
"chrony",
|
||||
]
|
||||
write_files = [
|
||||
{
|
||||
path = local.munge_stage_path
|
||||
encoding = "b64"
|
||||
content = local.munge_key_b64
|
||||
permissions = "0400"
|
||||
},
|
||||
{
|
||||
path = "/etc/slurm/slurm.conf"
|
||||
content = "${local.slurm_conf}\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
{
|
||||
path = "/etc/exports"
|
||||
content = "/home ${var.cluster_cidr}(rw,sync,no_subtree_check,root_squash)\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
]
|
||||
bootcmd = local.egress_bootcmd
|
||||
runcmd = concat(local.hosts_runcmd, local.munge_runcmd, [
|
||||
"mkdir -p /var/lib/slurm/slurmctld /var/log/slurm",
|
||||
"chown slurm:slurm /var/lib/slurm/slurmctld /var/log/slurm",
|
||||
"systemctl enable --now nfs-kernel-server",
|
||||
"exportfs -ra",
|
||||
"systemctl enable slurmctld",
|
||||
"systemctl restart slurmctld",
|
||||
])
|
||||
}
|
||||
|
||||
compute_cloud_config = [for i in range(var.compute_node_count) : {
|
||||
hostname = local.compute_hostnames[i]
|
||||
prefer_fqdn_over_hostname = false
|
||||
package_update = true
|
||||
packages = [
|
||||
"slurmd",
|
||||
"munge",
|
||||
"nfs-common",
|
||||
"openmpi-bin",
|
||||
"chrony",
|
||||
]
|
||||
bootcmd = local.egress_bootcmd
|
||||
write_files = [
|
||||
{
|
||||
path = local.munge_stage_path
|
||||
encoding = "b64"
|
||||
content = local.munge_key_b64
|
||||
permissions = "0400"
|
||||
},
|
||||
{
|
||||
path = "/etc/default/slurmd"
|
||||
content = "SLURMD_OPTIONS=\"--conf-server ${local.head_ip}:6817\"\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
{
|
||||
path = "/etc/systemd/system/slurmd.service.d/90-retry.conf"
|
||||
content = "[Unit]\nStartLimitIntervalSec=0\n\n[Service]\nRestart=on-failure\nRestartSec=15\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
]
|
||||
mounts = [
|
||||
["${local.head_ip}:/home", "/home", "nfs", "defaults,nofail,_netdev,x-systemd.automount", "0", "0"],
|
||||
]
|
||||
runcmd = concat(local.hosts_runcmd, local.munge_runcmd, [
|
||||
"mkdir -p /var/lib/slurm/slurmd /var/log/slurm",
|
||||
"chown slurm:slurm /var/lib/slurm/slurmd /var/log/slurm",
|
||||
// Best effort early mount of the shared home, so jobs submitted right
|
||||
// after the cluster forms never land on the local directory before the
|
||||
// automount first triggers.
|
||||
"systemctl start home.mount 2>/dev/null || true",
|
||||
// The retry drop-in keeps slurmd re-launching until the controller
|
||||
// answers: the cluster nodes boot in parallel and the first attempts
|
||||
// may precede slurmctld.
|
||||
"systemctl daemon-reload",
|
||||
"systemctl enable slurmd",
|
||||
"systemctl restart slurmd || true",
|
||||
])
|
||||
}]
|
||||
|
||||
// Consumer and management users, shared by every node. Fixed uids: Slurm
|
||||
// identifies users by uid across the cluster, so the consumer must exist
|
||||
// everywhere with the same one. The consumer has no sudo (it is a cluster
|
||||
// user, not a machine owner) and its password exists only on the head node
|
||||
// (applied over SSH, never through user data). merge directives make
|
||||
// cloud-init append to the node lists instead of replacing them.
|
||||
credentials_cloud_config = {
|
||||
merge_how = [
|
||||
{ name = "list", settings = ["append"] },
|
||||
{ name = "dict", settings = ["no_replace", "recurse_list"] },
|
||||
]
|
||||
ssh_pwauth = true
|
||||
users = [
|
||||
"default",
|
||||
{
|
||||
name = var.username
|
||||
uid = 2000
|
||||
shell = "/bin/bash"
|
||||
lock_passwd = false
|
||||
groups = "users"
|
||||
},
|
||||
{
|
||||
name = var.admin_username
|
||||
uid = 2001
|
||||
shell = "/bin/bash"
|
||||
lock_passwd = true
|
||||
sudo = "ALL=(ALL) NOPASSWD:ALL"
|
||||
groups = "users, sudo"
|
||||
ssh_authorized_keys = [trimspace(tls_private_key.admin.public_key_openssh)]
|
||||
},
|
||||
// The munge and slurm system users are created before the packages so
|
||||
// they get the SAME uid on every node: the packages otherwise allocate
|
||||
// them dynamically, and Slurm authenticates controller-to-daemon
|
||||
// messages by uid ("Security violation, slurm message from uid N"
|
||||
// across nodes when they diverge). Low fixed uids: the dynamic system
|
||||
// range is allocated downward from 999, so the high 900s are already
|
||||
// taken on the stock image. Listed last so a failure here can never
|
||||
// block the login users above.
|
||||
{
|
||||
name = "munge"
|
||||
uid = 601
|
||||
system = true
|
||||
shell = "/usr/sbin/nologin"
|
||||
homedir = "/var/lib/munge"
|
||||
no_create_home = true
|
||||
lock_passwd = true
|
||||
},
|
||||
{
|
||||
name = "slurm"
|
||||
uid = 602
|
||||
system = true
|
||||
shell = "/usr/sbin/nologin"
|
||||
homedir = "/var/lib/slurm"
|
||||
no_create_home = true
|
||||
lock_passwd = true
|
||||
},
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
data "cloudinit_config" "head" {
|
||||
gzip = false
|
||||
base64_encode = true
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.head_cloud_config)])
|
||||
}
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.credentials_cloud_config)])
|
||||
}
|
||||
}
|
||||
|
||||
data "cloudinit_config" "compute" {
|
||||
count = var.compute_node_count
|
||||
gzip = false
|
||||
base64_encode = true
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.compute_cloud_config[count.index])])
|
||||
}
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.credentials_cloud_config)])
|
||||
}
|
||||
}
|
||||
79
hpc/ovh/credentials.tf
Normal file
79
hpc/ovh/credentials.tf
Normal file
@ -0,0 +1,79 @@
|
||||
// Consumer and management credentials are generated here, never injected by
|
||||
// the backend. The consumer password opens SSH on the head node only (the
|
||||
// single entry point); the management keypair opens every node. The Vault
|
||||
// layout follows the existing contract:
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/{username} password + head node access
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/admin ssh keypair + admin username
|
||||
// Only the Vault path of the consumer entry is exposed as an output.
|
||||
|
||||
resource "random_bytes" "munge_key" {
|
||||
length = 128
|
||||
}
|
||||
|
||||
resource "tls_private_key" "admin" {
|
||||
algorithm = "ED25519"
|
||||
}
|
||||
|
||||
// Regenerated whenever the rotation marker changes; the special set avoids
|
||||
// shell quoting issues in the password application command.
|
||||
resource "random_password" "consumer" {
|
||||
length = 24
|
||||
special = true
|
||||
override_special = "-_=+"
|
||||
|
||||
keepers = {
|
||||
rotation = var.credentials_version
|
||||
}
|
||||
}
|
||||
|
||||
// Applies the consumer password on the head node, on day 0 and on every
|
||||
// rotation. The password never travels through user data, so a rotation
|
||||
// never touches the instances: this resource is replaced (triggered by the
|
||||
// rotation marker), re-runs chpasswd over SSH with the management key, and
|
||||
// the Vault entry below is updated in the same apply.
|
||||
resource "terraform_data" "consumer_password" {
|
||||
// Re-applied on rotation AND whenever the head node is recreated: a fresh
|
||||
// instance boots without the consumer password (it never travels through
|
||||
// user data).
|
||||
triggers_replace = [var.credentials_version, ovh_cloud_project_instance.head.id]
|
||||
|
||||
connection {
|
||||
type = "ssh"
|
||||
host = local.head_public_ip
|
||||
user = var.admin_username
|
||||
private_key = tls_private_key.admin.private_key_openssh
|
||||
timeout = "15m"
|
||||
}
|
||||
|
||||
provisioner "remote-exec" {
|
||||
inline = [
|
||||
"cloud-init status --wait > /dev/null || true",
|
||||
"echo '${var.username}:${random_password.consumer.result}' | sudo chpasswd",
|
||||
]
|
||||
}
|
||||
|
||||
depends_on = [ovh_cloud_project_instance.head]
|
||||
}
|
||||
|
||||
resource "vault_kv_secret_v2" "consumer" {
|
||||
mount = "secret"
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/${var.username}"
|
||||
data_json = jsonencode({
|
||||
username = var.username
|
||||
password = random_password.consumer.result
|
||||
host = local.head_public_ip
|
||||
port = 22
|
||||
})
|
||||
|
||||
depends_on = [terraform_data.consumer_password]
|
||||
}
|
||||
|
||||
resource "vault_kv_secret_v2" "admin" {
|
||||
mount = "secret"
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/admin"
|
||||
data_json = jsonencode({
|
||||
username = var.admin_username
|
||||
ssh_private_key = tls_private_key.admin.private_key_openssh
|
||||
ssh_public_key = trimspace(tls_private_key.admin.public_key_openssh)
|
||||
})
|
||||
}
|
||||
181
hpc/ovh/main.tf
Normal file
181
hpc/ovh/main.tf
Normal file
@ -0,0 +1,181 @@
|
||||
// HPC cluster: one head node (the single entry point, public SSH + private
|
||||
// cluster network) and N compute nodes (private network only, internet egress
|
||||
// through an OVH gateway for package installation). The cloud provider
|
||||
// credentials and the project (OVH_CLOUD_PROJECT_SERVICE) reach the provider
|
||||
// from the runner environment, never from module inputs.
|
||||
|
||||
data "ovh_cloud_project_flavors" "head" {
|
||||
service_name = var.project_id
|
||||
name_filter = var.head_flavor
|
||||
region = var.region
|
||||
}
|
||||
|
||||
data "ovh_cloud_project_flavors" "compute" {
|
||||
service_name = var.project_id
|
||||
name_filter = var.compute_flavor
|
||||
region = var.region
|
||||
}
|
||||
|
||||
data "ovh_cloud_project_images" "all" {
|
||||
service_name = var.project_id
|
||||
region = var.region
|
||||
}
|
||||
|
||||
locals {
|
||||
selected_image = one([for m in data.ovh_cloud_project_images.all.images : m if m.name == var.image])
|
||||
selected_head_flavor = one(data.ovh_cloud_project_flavors.head.flavors)
|
||||
selected_compute_flavor = one(data.ovh_cloud_project_flavors.compute.flavors)
|
||||
|
||||
head_hostname = "${var.name_prefix}-head-${var.deployment_id}"
|
||||
head_ip = cidrhost(var.cluster_cidr, 50)
|
||||
|
||||
compute_hostnames = [for i in range(var.compute_node_count) : "${var.name_prefix}-compute${i + 1}-${var.deployment_id}"]
|
||||
compute_ips = [for i in range(var.compute_node_count) : cidrhost(var.cluster_cidr, 60 + i)]
|
||||
}
|
||||
|
||||
// Private cluster network (vRack) with one subnet. Fixed addresses are
|
||||
// assigned outside the DHCP pool; DHCP stays enabled so the instances
|
||||
// configure their private interface from the OpenStack metadata.
|
||||
resource "ovh_cloud_project_network_private" "cluster" {
|
||||
service_name = var.project_id
|
||||
name = "${var.name_prefix}-hpc-${var.deployment_id}"
|
||||
regions = [var.region]
|
||||
vlan_id = var.vlan_id
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_network_private_subnet" "cluster" {
|
||||
service_name = var.project_id
|
||||
network_id = ovh_cloud_project_network_private.cluster.id
|
||||
region = var.region
|
||||
network = var.cluster_cidr
|
||||
// The allocation pool must cover the fixed addresses: the OVH API refuses
|
||||
// an instance ip outside it ("instance ip must be in subnet allocation
|
||||
// pool"). The DHCP agent ports allocate from the start of the pool, so the
|
||||
// fixed addresses (head at 50, compute nodes from 60) stay clear of them:
|
||||
// an instance on a DHCP-port address is silently canceled by the cloud.
|
||||
start = cidrhost(var.cluster_cidr, 10)
|
||||
end = cidrhost(var.cluster_cidr, 200)
|
||||
dhcp = true
|
||||
}
|
||||
|
||||
locals {
|
||||
openstack_network_id = one([
|
||||
for r in ovh_cloud_project_network_private.cluster.regions_attributes : r.openstackid if r.region == var.region
|
||||
])
|
||||
}
|
||||
|
||||
// Internet egress for the whole cluster: the gateway attaches to the
|
||||
// cluster subnet and DHCP distributes it as the default route. A NAT
|
||||
// through a dual-homed head node is not an option: the cloud's port
|
||||
// anti-spoofing silently drops the forwarded return traffic.
|
||||
resource "ovh_cloud_project_gateway" "egress" {
|
||||
service_name = var.project_id
|
||||
name = "${var.name_prefix}-hpc-gw-${var.deployment_id}"
|
||||
model = var.gateway_model
|
||||
region = var.region
|
||||
network_id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
|
||||
// One project SSH key per deployment, managed by the module so it is removed
|
||||
// on destroy. The instance level ssh_key_create is avoided on purpose: keys it
|
||||
// creates outlive the instance and collide on the next deployment.
|
||||
resource "ovh_cloud_project_ssh_key" "admin" {
|
||||
service_name = var.project_id
|
||||
name = "${var.name_prefix}-${var.deployment_id}-key"
|
||||
public_key = trimspace(tls_private_key.admin.public_key_openssh)
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_instance" "head" {
|
||||
service_name = var.project_id
|
||||
region = var.region
|
||||
billing_period = "hourly"
|
||||
name = local.head_hostname
|
||||
|
||||
boot_from {
|
||||
image_id = local.selected_image.id
|
||||
}
|
||||
|
||||
flavor {
|
||||
flavor_id = local.selected_head_flavor.id
|
||||
}
|
||||
|
||||
ssh_key {
|
||||
name = ovh_cloud_project_ssh_key.admin.name
|
||||
}
|
||||
|
||||
// The head node cannot sit on the public network: the cloud refuses an
|
||||
// instance attached to both the public network and a subnet served by a
|
||||
// gateway. Public reachability comes from a floating ip routed through
|
||||
// the gateway instead. The fixed address stays clear of the DHCP agent
|
||||
// ports at the start of the pool: an instance on a DHCP-port address is
|
||||
// silently canceled by the cloud, which also broke the floating ip
|
||||
// creation until the address moved.
|
||||
network {
|
||||
private {
|
||||
ip = local.head_ip
|
||||
floating_ip_create {
|
||||
description = "${var.name_prefix}-hpc-head-${var.deployment_id}"
|
||||
}
|
||||
network {
|
||||
id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
user_data = data.cloudinit_config.head.rendered
|
||||
|
||||
depends_on = [ovh_cloud_project_gateway.egress]
|
||||
}
|
||||
|
||||
// Resolves the public (floating) address of the head node: the instance
|
||||
// addresses only carry the fixed private ip, the floating ip is matched
|
||||
// through the port it is associated with.
|
||||
data "ovh_cloud_project_floatingips" "all" {
|
||||
service_name = var.project_id
|
||||
region_name = var.region
|
||||
|
||||
depends_on = [ovh_cloud_project_instance.head]
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_instance" "compute" {
|
||||
count = var.compute_node_count
|
||||
service_name = var.project_id
|
||||
region = var.region
|
||||
billing_period = "hourly"
|
||||
name = local.compute_hostnames[count.index]
|
||||
|
||||
boot_from {
|
||||
image_id = local.selected_image.id
|
||||
}
|
||||
|
||||
flavor {
|
||||
flavor_id = local.selected_compute_flavor.id
|
||||
}
|
||||
|
||||
ssh_key {
|
||||
name = ovh_cloud_project_ssh_key.admin.name
|
||||
}
|
||||
|
||||
network {
|
||||
private {
|
||||
ip = local.compute_ips[count.index]
|
||||
network {
|
||||
id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
user_data = data.cloudinit_config.compute[count.index].rendered
|
||||
|
||||
depends_on = [ovh_cloud_project_gateway.egress]
|
||||
}
|
||||
|
||||
locals {
|
||||
head_public_ip = one([
|
||||
for f in data.ovh_cloud_project_floatingips.all.cloud_project_floatingips : f.ip
|
||||
if try(f.associated_entity.ip, "") == local.head_ip
|
||||
])
|
||||
}
|
||||
28
hpc/ovh/outputs.tf
Normal file
28
hpc/ovh/outputs.tf
Normal file
@ -0,0 +1,28 @@
|
||||
// Deliberately restricted to the entry point: the response payload returned
|
||||
// to the backend must carry strictly the head node access information, never
|
||||
// the compute node details.
|
||||
|
||||
output "head_node_ip" {
|
||||
description = "Public IPv4 address of the head node, the single entry point of the cluster"
|
||||
value = local.head_public_ip
|
||||
}
|
||||
|
||||
output "head_node_hostname" {
|
||||
description = "Hostname of the head node"
|
||||
value = local.head_hostname
|
||||
}
|
||||
|
||||
output "ssh_port" {
|
||||
description = "SSH port of the head node"
|
||||
value = 22
|
||||
}
|
||||
|
||||
output "vmIps" {
|
||||
description = "Entry point address list, kept for compatibility with the single VM notification flow"
|
||||
value = [local.head_public_ip]
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
description = "Vault path of the generated consumer credentials for the head node"
|
||||
value = "secret/data/${vault_kv_secret_v2.consumer.name}"
|
||||
}
|
||||
102
hpc/ovh/variables.tf
Normal file
102
hpc/ovh/variables.tf
Normal file
@ -0,0 +1,102 @@
|
||||
variable "project_id" {
|
||||
description = "OVH public cloud project id (service_name). Defaults to the OVH_CLOUD_PROJECT_SERVICE variable of the runner environment, derived from the cloud account configuration."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "OVH region for all cluster instances and the private network, for example SBG5 or GRA11"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "deployment_id" {
|
||||
description = "Unique deployment identifier, used to build resource names"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_account" {
|
||||
description = "Cloud account (environment) name, second segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "requester_unique_id" {
|
||||
description = "Requester unique id, third segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "name_prefix" {
|
||||
description = "Prefix for all resource names"
|
||||
type = string
|
||||
default = "simpl"
|
||||
}
|
||||
|
||||
variable "username" {
|
||||
description = "Consumer username on the head node, last segment of the Vault credentials path. The user exists on every node with the same uid (Slurm runs jobs under it), but can only log in on the head node."
|
||||
type = string
|
||||
default = "simpl_user"
|
||||
}
|
||||
|
||||
variable "admin_username" {
|
||||
description = "Management username created on all nodes with SSH key access only"
|
||||
type = string
|
||||
default = "simpladmin"
|
||||
}
|
||||
|
||||
variable "credentials_version" {
|
||||
description = "Opaque rotation marker for the consumer credential. Changing the value regenerates the consumer password, applies it on the head node over SSH and republishes the fresh credential to Vault in the same apply, without touching the cluster nodes. Managed by the credential renewal workflow, not by the deployment script author."
|
||||
type = string
|
||||
default = "1"
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
description = "Image name for every cluster node, must be a cloud-init enabled image"
|
||||
type = string
|
||||
default = "Ubuntu 24.04"
|
||||
}
|
||||
|
||||
variable "head_flavor" {
|
||||
description = "Instance flavor for the head node, for example b3-8"
|
||||
type = string
|
||||
default = "b3-8"
|
||||
}
|
||||
|
||||
variable "compute_flavor" {
|
||||
description = "Instance flavor for the compute nodes, for example b3-8"
|
||||
type = string
|
||||
default = "b3-8"
|
||||
}
|
||||
|
||||
variable "compute_node_count" {
|
||||
description = "Number of compute nodes in the cluster"
|
||||
type = number
|
||||
default = 2
|
||||
|
||||
validation {
|
||||
condition = var.compute_node_count >= 1
|
||||
error_message = "The cluster needs at least one compute node."
|
||||
}
|
||||
}
|
||||
|
||||
variable "compute_cpus" {
|
||||
description = "Number of schedulable CPUs per compute node, declared to the Slurm controller. Must not exceed the vCPUs of compute_flavor."
|
||||
type = number
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "cluster_cidr" {
|
||||
description = "CIDR of the private cluster network. The head node takes the 50th host address, compute nodes take the 60th onward (lower pool addresses are left to the DHCP agent ports)."
|
||||
type = string
|
||||
default = "192.168.100.0/24"
|
||||
}
|
||||
|
||||
variable "vlan_id" {
|
||||
description = "vRack VLAN id for the private cluster network, must be unused in the project"
|
||||
type = number
|
||||
default = 212
|
||||
}
|
||||
|
||||
variable "gateway_model" {
|
||||
description = "Model (size) of the cloud gateway providing internet egress to the cluster"
|
||||
type = string
|
||||
default = "s"
|
||||
}
|
||||
24
hpc/ovh/versions.tf
Normal file
24
hpc/ovh/versions.tf
Normal file
@ -0,0 +1,24 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
ovh = {
|
||||
source = "ovh/ovh"
|
||||
version = "2.3.0"
|
||||
}
|
||||
vault = {
|
||||
source = "hashicorp/vault"
|
||||
version = "4.8.0"
|
||||
}
|
||||
random = {
|
||||
source = "hashicorp/random"
|
||||
version = "3.7.2"
|
||||
}
|
||||
tls = {
|
||||
source = "hashicorp/tls"
|
||||
version = "4.0.6"
|
||||
}
|
||||
cloudinit = {
|
||||
source = "hashicorp/cloudinit"
|
||||
version = "2.3.7"
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Reference in New Issue
Block a user