Compare commits
No commits in common. "master" and "3.2.0-F-feature-simpl-31668.650.242a7609" have entirely different histories.
master
...
3.2.0-F-fe
@ -1,65 +0,0 @@
|
||||
# dbaas module
|
||||
|
||||
Provisions a managed database and generates the consumer access credentials. One submodule per cloud provider, selected by the provisioner from the cloud provider of the provisioning request:
|
||||
|
||||
- `ovh`: an OVHcloud Managed PostgreSQL cluster (`ovh_cloud_project_database`) with one initial database and one application user, public by default, optionally attached to an existing vRack private network
|
||||
|
||||
## Contract
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `deployment_id` | Unique deployment identifier, used in resource names |
|
||||
| `cloud_account` | Cloud account (environment) name, second segment of the Vault credentials path, filled by the provisioner from the cloudAccount header via the `{CLOUD_ACCOUNT}` placeholder |
|
||||
| `requester_unique_id` | Requester unique id, third segment of the Vault credentials path, filled by the provisioner from the requesterUniqueId header via the `{REQUESTER_UNIQUE_ID}` placeholder |
|
||||
| `username` | Database user name, defaults to `simpl_app_user`, last segment of the Vault credentials path |
|
||||
| `credentials_version` | Opaque rotation marker for the database user credential, defaults to `1`. Changing the value makes OVHcloud regenerate the user password and republishes the fresh credential to Vault. Managed by the credential renewal workflow, not by the deployment script author |
|
||||
| `name_prefix` | Prefix for resource names, defaults to `simpl` |
|
||||
| `region` | OVH macro region for the database nodes (for example SBG, GRA, DE), not the datacenter suffixed region used for instances |
|
||||
| `project_id` | Optional override of the OVH project; by default the provider resolves it from the `OVH_CLOUD_PROJECT_SERVICE` variable of the runner environment, which the provisioner derives from the `OVH_PROJECT_ID` key of the cloud account configuration |
|
||||
| `engine_version`, `plan`, `flavor` | PostgreSQL version and sizing of the managed offer |
|
||||
| `database_name` | Name of the initial application database, defaults to `simpl_app_db` |
|
||||
| `private_network_id`, `private_subnet_id` | Optional vRack attachment, both null for a public deployment |
|
||||
| `allowed_ips` | CIDR blocks allowed to reach a public deployment |
|
||||
|
||||
Cloud credentials are never module inputs, they reach the provider from the runner environment.
|
||||
|
||||
Credentials, following the existing Vault contract: the OVHcloud managed database password is always generated server side (it cannot be set to an arbitrary value), read from the computed attribute of the user resource and written by the module to Vault: `secret/{provider}/{cloud_account}/{requester_unique_id}/{username}` with the keys username, password, host, port, database and a ready to use connection uri. No secret material is ever part of an output or of the user body. There is no admin sibling entry: day 2 credential renewal follows the same Terraform rotation flow as the other catalog modules, changing the `credentials_version` input, which makes OVHcloud regenerate the password server side and republishes the Vault entry in the same apply. The Vault provider configuration is generated by the provisioner next to the deployment script, and it authenticates through the Vault Kubernetes auth method with the service account of the Terraform runner, against a Vault role configured at installation time.
|
||||
|
||||
Outputs:
|
||||
|
||||
| Output | Description |
|
||||
|---|---|
|
||||
| `database_endpoint` | Service endpoint (hostname) of the PostgreSQL instance |
|
||||
| `database_port` | Port of the PostgreSQL instance |
|
||||
| `database_name` | Name of the initial database |
|
||||
| `database_username` | Database user name |
|
||||
| `database_cluster_id` | Id of the managed database cluster |
|
||||
| `database_user_id` | Id of the database user |
|
||||
| `credentials_ref` | Vault path of the consumer credentials entry |
|
||||
|
||||
## Usage
|
||||
|
||||
The provisioner rewrites the `{DBAAS_SOURCE}` placeholder to this module's location for the cloud provider of the request (each catalog module has a placeholder derived from its directory name), so a deployment script only declares:
|
||||
|
||||
```hcl
|
||||
module "dbaas" {
|
||||
source = "{DBAAS_SOURCE}"
|
||||
deployment_id = "{UUID}"
|
||||
region = "<region>"
|
||||
cloud_account = "{CLOUD_ACCOUNT}"
|
||||
requester_unique_id = "{REQUESTER_UNIQUE_ID}"
|
||||
|
||||
plan = "essential"
|
||||
flavor = "db1-4"
|
||||
}
|
||||
|
||||
output "database_endpoint" {
|
||||
value = module.dbaas.database_endpoint
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
value = module.dbaas.credentials_ref
|
||||
}
|
||||
```
|
||||
|
||||
The root file must re-export the module outputs it wants published to the deployment outputs.
|
||||
@ -1,28 +0,0 @@
|
||||
// The consumer credential is generated by OVHcloud (server side password)
|
||||
// and delivered only through Vault, never through outputs or the user body.
|
||||
// The Vault entry follows the existing contract:
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/{username}
|
||||
// Only the Vault path of the entry is exposed as an output. There is no
|
||||
// admin sibling entry: day 2 credential renewal follows the same Terraform
|
||||
// rotation flow as the other catalog modules (the renewal workflow rewrites
|
||||
// a rotation marker next to the deployment script and the apply regenerates
|
||||
// the password, republishing this entry); the rotation marker input is
|
||||
// added by the credential renewal ticket.
|
||||
|
||||
locals {
|
||||
database_host = one([for e in ovh_cloud_project_database.this.endpoints : e.domain if e.component == "postgresql"])
|
||||
database_port = one([for e in ovh_cloud_project_database.this.endpoints : e.port if e.component == "postgresql"])
|
||||
}
|
||||
|
||||
resource "vault_kv_secret_v2" "consumer" {
|
||||
mount = "secret"
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/${var.username}"
|
||||
data_json = jsonencode({
|
||||
username = ovh_cloud_project_database_postgresql_user.this.name
|
||||
password = ovh_cloud_project_database_postgresql_user.this.password
|
||||
host = local.database_host
|
||||
port = local.database_port
|
||||
database = ovh_cloud_project_database_database.this.name
|
||||
uri = "postgres://${ovh_cloud_project_database_postgresql_user.this.name}:${ovh_cloud_project_database_postgresql_user.this.password}@${local.database_host}:${local.database_port}/${ovh_cloud_project_database_database.this.name}?sslmode=require"
|
||||
})
|
||||
}
|
||||
@ -1,50 +0,0 @@
|
||||
// OVHcloud Managed PostgreSQL. The cloud provider credentials and the
|
||||
// project (OVH_CLOUD_PROJECT_SERVICE) reach the provider from the runner
|
||||
// environment, never from module inputs.
|
||||
|
||||
resource "ovh_cloud_project_database" "this" {
|
||||
service_name = var.project_id
|
||||
description = "${var.name_prefix}-postgresql-${var.deployment_id}"
|
||||
engine = "postgresql"
|
||||
version = var.engine_version
|
||||
plan = var.plan
|
||||
flavor = var.flavor
|
||||
|
||||
nodes {
|
||||
region = var.region
|
||||
network_id = var.private_network_id
|
||||
subnet_id = var.private_subnet_id
|
||||
}
|
||||
|
||||
dynamic "ip_restrictions" {
|
||||
for_each = var.allowed_ips
|
||||
content {
|
||||
ip = ip_restrictions.value
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_database_database" "this" {
|
||||
service_name = ovh_cloud_project_database.this.service_name
|
||||
engine = ovh_cloud_project_database.this.engine
|
||||
cluster_id = ovh_cloud_project_database.this.id
|
||||
name = var.database_name
|
||||
}
|
||||
|
||||
// The user password is ALWAYS generated server side by OVHcloud, it cannot
|
||||
// be set to an arbitrary value: it is read from the computed .password
|
||||
// attribute and delivered only through the Vault entry. password_reset is
|
||||
// the provider's native rotation marker: any change to credentials_version
|
||||
// makes OVHcloud regenerate the password in place, and the Vault entry is
|
||||
// updated with the fresh credential in the same apply.
|
||||
resource "ovh_cloud_project_database_postgresql_user" "this" {
|
||||
service_name = ovh_cloud_project_database.this.service_name
|
||||
cluster_id = ovh_cloud_project_database.this.id
|
||||
name = var.username
|
||||
password_reset = var.credentials_version
|
||||
|
||||
// Serialize operations against the cluster: OVH DBaaS processes one
|
||||
// operation at a time per cluster, and concurrent destroys (user +
|
||||
// database in parallel) intermittently fail with API 500s.
|
||||
depends_on = [ovh_cloud_project_database_database.this]
|
||||
}
|
||||
@ -1,34 +0,0 @@
|
||||
output "database_endpoint" {
|
||||
description = "Service endpoint (hostname) of the PostgreSQL instance"
|
||||
value = local.database_host
|
||||
}
|
||||
|
||||
output "database_port" {
|
||||
description = "Port of the PostgreSQL instance"
|
||||
value = local.database_port
|
||||
}
|
||||
|
||||
output "database_name" {
|
||||
description = "Name of the initial database"
|
||||
value = ovh_cloud_project_database_database.this.name
|
||||
}
|
||||
|
||||
output "database_username" {
|
||||
description = "Database user name"
|
||||
value = ovh_cloud_project_database_postgresql_user.this.name
|
||||
}
|
||||
|
||||
output "database_cluster_id" {
|
||||
description = "Id of the managed database cluster"
|
||||
value = ovh_cloud_project_database.this.id
|
||||
}
|
||||
|
||||
output "database_user_id" {
|
||||
description = "Id of the database user"
|
||||
value = ovh_cloud_project_database_postgresql_user.this.id
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
description = "Vault path of the consumer credentials entry"
|
||||
value = "secret/data/${vault_kv_secret_v2.consumer.name}"
|
||||
}
|
||||
@ -1,85 +0,0 @@
|
||||
variable "project_id" {
|
||||
description = "OVH public cloud project id (service_name). Defaults to the OVH_CLOUD_PROJECT_SERVICE variable of the runner environment, derived from the cloud account configuration."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "OVH region for the database nodes. Managed databases use the macro region code (for example SBG, GRA, DE), not the datacenter suffixed region (SBG5) used for instances."
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "deployment_id" {
|
||||
description = "Unique deployment identifier, used to build resource names"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_account" {
|
||||
description = "Cloud account (environment) name, second segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "requester_unique_id" {
|
||||
description = "Requester unique id, third segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "name_prefix" {
|
||||
description = "Prefix for all resource names"
|
||||
type = string
|
||||
default = "simpl"
|
||||
}
|
||||
|
||||
variable "username" {
|
||||
description = "Database user name, last segment of the Vault credentials path"
|
||||
type = string
|
||||
default = "simpl_app_user"
|
||||
}
|
||||
|
||||
variable "credentials_version" {
|
||||
description = "Opaque rotation marker for the database user credential. Changing the value makes OVHcloud regenerate the user password server side and republishes the fresh credential to Vault in the same apply. Managed by the credential renewal workflow, not by the deployment script author."
|
||||
type = string
|
||||
default = "1"
|
||||
}
|
||||
|
||||
variable "engine_version" {
|
||||
description = "PostgreSQL engine version to provision"
|
||||
type = string
|
||||
default = "16"
|
||||
}
|
||||
|
||||
variable "plan" {
|
||||
description = "Service plan of the managed offer: essential, business or production"
|
||||
type = string
|
||||
default = "essential"
|
||||
}
|
||||
|
||||
variable "flavor" {
|
||||
description = "Node flavor (compute/memory sizing) for the database cluster, for example db1-4"
|
||||
type = string
|
||||
default = "db1-4"
|
||||
}
|
||||
|
||||
variable "database_name" {
|
||||
description = "Name of the initial application database to create on the cluster"
|
||||
type = string
|
||||
default = "simpl_app_db"
|
||||
}
|
||||
|
||||
variable "private_network_id" {
|
||||
description = "OpenStack private network (vRack) id the database nodes should be attached to, null for a public deployment"
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "private_subnet_id" {
|
||||
description = "Subnet id (within private_network_id) used by the database nodes, null for a public deployment"
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "allowed_ips" {
|
||||
description = "CIDR blocks allowed to reach the cluster (IP restrictions), only relevant for public deployments"
|
||||
type = list(string)
|
||||
default = []
|
||||
}
|
||||
@ -1,12 +0,0 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
ovh = {
|
||||
source = "ovh/ovh"
|
||||
version = "2.3.0"
|
||||
}
|
||||
vault = {
|
||||
source = "hashicorp/vault"
|
||||
version = "4.8.0"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,70 +0,0 @@
|
||||
# hpc module
|
||||
|
||||
Provisions an HPC cluster and generates the consumer access credentials. One submodule per cloud provider, selected by the provisioner from the cloud provider of the provisioning request:
|
||||
|
||||
- `ovh`: one head node (Slurm controller, munge authority, NFS server, consumer entry point) plus N compute nodes (Slurm execution daemon, OpenMPI, NFS client) on a dedicated private network (vRack). All nodes are private only: internet egress goes through a managed cloud gateway attached to the cluster subnet, and the head node is reached from outside through a floating ip routed by the same gateway. The cluster bootstraps entirely from per-role cloud-init at first boot (Slurm configless, per-deployment munge key, shared `/home` over NFS, fixed uids cluster-wide) with no manual step.
|
||||
|
||||
## Contract
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `deployment_id` | Unique deployment identifier, used in resource names |
|
||||
| `cloud_account` | Cloud account (environment) name, second segment of the Vault credentials path, filled by the provisioner from the cloudAccount header via the `{CLOUD_ACCOUNT}` placeholder |
|
||||
| `requester_unique_id` | Requester unique id, third segment of the Vault credentials path, filled by the provisioner from the requesterUniqueId header via the `{REQUESTER_UNIQUE_ID}` placeholder |
|
||||
| `username` | Consumer username on the head node, defaults to `simpl_user`, last segment of the Vault credentials path. The user exists on every node with the same uid (Slurm runs jobs under it) but can only log in on the head node |
|
||||
| `admin_username` | Management username created on all nodes with SSH key access only, defaults to `simpladmin` |
|
||||
| `credentials_version` | Opaque rotation marker for the consumer credential, defaults to `1`. Changing the value regenerates the consumer password, applies it on the head node over SSH and republishes the fresh credential to Vault in the same apply, without touching any instance. Managed by the credential renewal workflow, not by the deployment script author |
|
||||
| `name_prefix` | Prefix for resource names, defaults to `simpl` |
|
||||
| `region` | OVH region for the instances and the private network, for example SBG5 |
|
||||
| `project_id` | OVH project id. Unlike the other catalog modules this one effectively needs it: its flavors/images/floatingips data sources require an explicit service_name (no `OVH_CLOUD_PROJECT_SERVICE` fallback for an explicit null). When omitted, the module falls back to resolving the project from the credential (`ovh_cloud_projects`), which only works if the account API key is granted `GET /cloud/project`; the current platform account keys are not, so either pass the value in the deployment script or add that read grant when creating account keys |
|
||||
| `compute_node_count` | Number of compute nodes, defaults to 2 |
|
||||
| `head_flavor`, `compute_flavor` | Instance flavors per role, default `b3-8` |
|
||||
| `compute_cpus` | Schedulable CPUs per compute node declared to the Slurm controller, must not exceed the vCPUs of `compute_flavor`, defaults to 2 |
|
||||
| `image` | Cloud-init enabled image for every node, defaults to `Ubuntu 24.04` |
|
||||
| `cluster_cidr` | Private cluster network CIDR, defaults to `192.168.100.0/24`. The head node takes host 50, compute nodes host 60 onward; lower pool addresses stay free for the DHCP agent ports (an instance on a DHCP-port address is silently canceled by the cloud) |
|
||||
| `vlan_id` | vRack VLAN id for the private network, must be unused in the project, defaults to 212 |
|
||||
| `gateway_model` | Size of the cloud gateway providing the cluster egress, defaults to `s` |
|
||||
|
||||
Cloud credentials are never module inputs, they reach the provider from the runner environment.
|
||||
|
||||
Credentials, following the existing Vault contract: the module generates the consumer password and the management SSH keypair and writes them to Vault, `secret/{provider}/{cloud_account}/{requester_unique_id}/{username}` (keys username, password, host, port) and the `.../admin` sibling (keys username, ssh_private_key, ssh_public_key). The consumer password never travels through user data: it is applied on the head node over SSH, keyed on both the rotation marker and the head node instance id, so a recreated head node receives the credential again. Day 2 credential renewal follows the same Terraform rotation flow as the other catalog modules, changing the `credentials_version` input. No secret material is ever part of an output or of the user body. The Vault provider configuration is generated by the provisioner next to the deployment script, and it authenticates through the Vault Kubernetes auth method with the service account of the Terraform runner, against a Vault role configured at installation time.
|
||||
|
||||
Outputs, deliberately restricted to the entry point (the response payload returned to the backend carries strictly the head node access information, never the compute node details):
|
||||
|
||||
| Output | Description |
|
||||
|---|---|
|
||||
| `head_node_ip` | Public (floating) IPv4 address of the head node, the single entry point of the cluster |
|
||||
| `head_node_hostname` | Hostname of the head node |
|
||||
| `ssh_port` | SSH port of the head node |
|
||||
| `vmIps` | Entry point address list, kept for compatibility with the single VM notification flow |
|
||||
| `credentials_ref` | Vault path of the consumer credentials entry |
|
||||
|
||||
## Usage
|
||||
|
||||
The provisioner rewrites the `{HPC_SOURCE}` placeholder to this module's location for the cloud provider of the request (each catalog module has a placeholder derived from its directory name), so a deployment script only declares:
|
||||
|
||||
```hcl
|
||||
module "hpc" {
|
||||
source = "{HPC_SOURCE}"
|
||||
deployment_id = "{UUID}"
|
||||
region = "<region>"
|
||||
cloud_account = "{CLOUD_ACCOUNT}"
|
||||
requester_unique_id = "{REQUESTER_UNIQUE_ID}"
|
||||
|
||||
compute_node_count = 2
|
||||
}
|
||||
|
||||
output "head_node_ip" {
|
||||
value = module.hpc.head_node_ip
|
||||
}
|
||||
|
||||
output "vmIps" {
|
||||
value = module.hpc.vmIps
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
value = module.hpc.credentials_ref
|
||||
}
|
||||
```
|
||||
|
||||
The root file must re-export the module outputs it wants published to the deployment outputs.
|
||||
@ -1,245 +0,0 @@
|
||||
// Node bootstrap. Everything local to a node is applied by cloud-init at
|
||||
// first boot; the cluster forms without any manual step. Slurm runs in
|
||||
// configless mode: only the head node carries slurm.conf, compute nodes
|
||||
// fetch it from the controller. The munge key is generated per deployment
|
||||
// and delivered to every node through its user data.
|
||||
|
||||
locals {
|
||||
munge_key_b64 = random_bytes.munge_key.base64
|
||||
|
||||
// Written early by cloud-init to a staging path (the munge user does not
|
||||
// exist yet); runcmd installs it with the right owner after the packages.
|
||||
munge_stage_path = "/run/simpl/munge.key"
|
||||
|
||||
cluster_hosts = concat(
|
||||
["${local.head_ip} ${local.head_hostname}"],
|
||||
[for i in range(var.compute_node_count) : "${local.compute_ips[i]} ${local.compute_hostnames[i]}"],
|
||||
)
|
||||
|
||||
slurm_conf = join("\n", concat([
|
||||
"ClusterName=${lower(var.name_prefix)}-hpc",
|
||||
"SlurmctldHost=${local.head_hostname}(${local.head_ip})",
|
||||
"SlurmctldParameters=enable_configless",
|
||||
"AuthType=auth/munge",
|
||||
"SlurmUser=slurm",
|
||||
"CredType=cred/munge",
|
||||
"SelectType=select/cons_tres",
|
||||
"SelectTypeParameters=CR_Core",
|
||||
"ProctrackType=proctrack/linuxproc",
|
||||
"TaskPlugin=task/none",
|
||||
"ReturnToService=2",
|
||||
"SlurmctldPort=6817",
|
||||
"SlurmdPort=6818",
|
||||
"SrunPortRange=60001-60128",
|
||||
"StateSaveLocation=/var/lib/slurm/slurmctld",
|
||||
"SlurmdSpoolDir=/var/lib/slurm/slurmd",
|
||||
"SlurmctldLogFile=/var/log/slurm/slurmctld.log",
|
||||
"SlurmdLogFile=/var/log/slurm/slurmd.log",
|
||||
"MailProg=/bin/true",
|
||||
], [
|
||||
for i in range(var.compute_node_count) :
|
||||
"NodeName=${local.compute_hostnames[i]} NodeAddr=${local.compute_ips[i]} CPUs=${var.compute_cpus} State=UNKNOWN"
|
||||
], [
|
||||
"PartitionName=main Nodes=ALL Default=YES MaxTime=INFINITE State=UP",
|
||||
]))
|
||||
|
||||
hosts_runcmd = [for entry in local.cluster_hosts : "grep -qxF '${entry}' /etc/hosts || echo '${entry}' >> /etc/hosts"]
|
||||
|
||||
munge_runcmd = [
|
||||
"install -o munge -g munge -m 0400 ${local.munge_stage_path} /etc/munge/munge.key",
|
||||
"rm -f ${local.munge_stage_path}",
|
||||
"systemctl restart munge",
|
||||
]
|
||||
|
||||
// Every cluster node is on the private subnet only: the default route
|
||||
// comes from DHCP (the subnet gateway) but no name servers are provided,
|
||||
// so resolution is pinned to public resolvers. Runs on every boot and
|
||||
// before package installation; the last entry blocks until egress through
|
||||
// the gateway answers, capped at ~4 minutes.
|
||||
egress_bootcmd = [
|
||||
"mkdir -p /etc/systemd/resolved.conf.d",
|
||||
"printf '[Resolve]\\nDNS=1.1.1.1 8.8.8.8\\n' > /etc/systemd/resolved.conf.d/90-cluster.conf",
|
||||
"systemctl try-restart systemd-resolved || true",
|
||||
"for i in $(seq 1 48); do timeout 3 bash -c '</dev/tcp/archive.ubuntu.com/80' 2>/dev/null && break; sleep 5; done",
|
||||
]
|
||||
|
||||
head_cloud_config = {
|
||||
hostname = local.head_hostname
|
||||
prefer_fqdn_over_hostname = false
|
||||
package_update = true
|
||||
packages = [
|
||||
"slurmctld",
|
||||
"slurm-client",
|
||||
"munge",
|
||||
"nfs-kernel-server",
|
||||
"openmpi-bin",
|
||||
"chrony",
|
||||
]
|
||||
write_files = [
|
||||
{
|
||||
path = local.munge_stage_path
|
||||
encoding = "b64"
|
||||
content = local.munge_key_b64
|
||||
permissions = "0400"
|
||||
},
|
||||
{
|
||||
path = "/etc/slurm/slurm.conf"
|
||||
content = "${local.slurm_conf}\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
{
|
||||
path = "/etc/exports"
|
||||
content = "/home ${var.cluster_cidr}(rw,sync,no_subtree_check,root_squash)\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
]
|
||||
bootcmd = local.egress_bootcmd
|
||||
runcmd = concat(local.hosts_runcmd, local.munge_runcmd, [
|
||||
"mkdir -p /var/lib/slurm/slurmctld /var/log/slurm",
|
||||
"chown slurm:slurm /var/lib/slurm/slurmctld /var/log/slurm",
|
||||
"systemctl enable --now nfs-kernel-server",
|
||||
"exportfs -ra",
|
||||
"systemctl enable slurmctld",
|
||||
"systemctl restart slurmctld",
|
||||
])
|
||||
}
|
||||
|
||||
compute_cloud_config = [for i in range(var.compute_node_count) : {
|
||||
hostname = local.compute_hostnames[i]
|
||||
prefer_fqdn_over_hostname = false
|
||||
package_update = true
|
||||
packages = [
|
||||
"slurmd",
|
||||
"munge",
|
||||
"nfs-common",
|
||||
"openmpi-bin",
|
||||
"chrony",
|
||||
]
|
||||
bootcmd = local.egress_bootcmd
|
||||
write_files = [
|
||||
{
|
||||
path = local.munge_stage_path
|
||||
encoding = "b64"
|
||||
content = local.munge_key_b64
|
||||
permissions = "0400"
|
||||
},
|
||||
{
|
||||
path = "/etc/default/slurmd"
|
||||
content = "SLURMD_OPTIONS=\"--conf-server ${local.head_ip}:6817\"\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
{
|
||||
path = "/etc/systemd/system/slurmd.service.d/90-retry.conf"
|
||||
content = "[Unit]\nStartLimitIntervalSec=0\n\n[Service]\nRestart=on-failure\nRestartSec=15\n"
|
||||
permissions = "0644"
|
||||
},
|
||||
]
|
||||
mounts = [
|
||||
["${local.head_ip}:/home", "/home", "nfs", "defaults,nofail,_netdev,x-systemd.automount", "0", "0"],
|
||||
]
|
||||
runcmd = concat(local.hosts_runcmd, local.munge_runcmd, [
|
||||
"mkdir -p /var/lib/slurm/slurmd /var/log/slurm",
|
||||
"chown slurm:slurm /var/lib/slurm/slurmd /var/log/slurm",
|
||||
// Best effort early mount of the shared home, so jobs submitted right
|
||||
// after the cluster forms never land on the local directory before the
|
||||
// automount first triggers.
|
||||
"systemctl start home.mount 2>/dev/null || true",
|
||||
// The retry drop-in keeps slurmd re-launching until the controller
|
||||
// answers: the cluster nodes boot in parallel and the first attempts
|
||||
// may precede slurmctld.
|
||||
"systemctl daemon-reload",
|
||||
"systemctl enable slurmd",
|
||||
"systemctl restart slurmd || true",
|
||||
])
|
||||
}]
|
||||
|
||||
// Consumer and management users, shared by every node. Fixed uids: Slurm
|
||||
// identifies users by uid across the cluster, so the consumer must exist
|
||||
// everywhere with the same one. The consumer has no sudo (it is a cluster
|
||||
// user, not a machine owner) and its password exists only on the head node
|
||||
// (applied over SSH, never through user data). merge directives make
|
||||
// cloud-init append to the node lists instead of replacing them.
|
||||
credentials_cloud_config = {
|
||||
merge_how = [
|
||||
{ name = "list", settings = ["append"] },
|
||||
{ name = "dict", settings = ["no_replace", "recurse_list"] },
|
||||
]
|
||||
ssh_pwauth = true
|
||||
users = [
|
||||
"default",
|
||||
{
|
||||
name = var.username
|
||||
uid = 2000
|
||||
shell = "/bin/bash"
|
||||
lock_passwd = false
|
||||
groups = "users"
|
||||
},
|
||||
{
|
||||
name = var.admin_username
|
||||
uid = 2001
|
||||
shell = "/bin/bash"
|
||||
lock_passwd = true
|
||||
sudo = "ALL=(ALL) NOPASSWD:ALL"
|
||||
groups = "users, sudo"
|
||||
ssh_authorized_keys = [trimspace(tls_private_key.admin.public_key_openssh)]
|
||||
},
|
||||
// The munge and slurm system users are created before the packages so
|
||||
// they get the SAME uid on every node: the packages otherwise allocate
|
||||
// them dynamically, and Slurm authenticates controller-to-daemon
|
||||
// messages by uid ("Security violation, slurm message from uid N"
|
||||
// across nodes when they diverge). Low fixed uids: the dynamic system
|
||||
// range is allocated downward from 999, so the high 900s are already
|
||||
// taken on the stock image. Listed last so a failure here can never
|
||||
// block the login users above.
|
||||
{
|
||||
name = "munge"
|
||||
uid = 601
|
||||
system = true
|
||||
shell = "/usr/sbin/nologin"
|
||||
homedir = "/var/lib/munge"
|
||||
no_create_home = true
|
||||
lock_passwd = true
|
||||
},
|
||||
{
|
||||
name = "slurm"
|
||||
uid = 602
|
||||
system = true
|
||||
shell = "/usr/sbin/nologin"
|
||||
homedir = "/var/lib/slurm"
|
||||
no_create_home = true
|
||||
lock_passwd = true
|
||||
},
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
data "cloudinit_config" "head" {
|
||||
gzip = false
|
||||
base64_encode = true
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.head_cloud_config)])
|
||||
}
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.credentials_cloud_config)])
|
||||
}
|
||||
}
|
||||
|
||||
data "cloudinit_config" "compute" {
|
||||
count = var.compute_node_count
|
||||
gzip = false
|
||||
base64_encode = true
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.compute_cloud_config[count.index])])
|
||||
}
|
||||
|
||||
part {
|
||||
content_type = "text/cloud-config"
|
||||
content = join("\n", ["#cloud-config", yamlencode(local.credentials_cloud_config)])
|
||||
}
|
||||
}
|
||||
@ -1,84 +0,0 @@
|
||||
// Consumer and management credentials are generated here, never injected by
|
||||
// the backend. The consumer password opens SSH on the head node only (the
|
||||
// single entry point); the management keypair opens every node. The Vault
|
||||
// layout follows the existing contract:
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/{username} password + head node access
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/admin ssh keypair + admin username
|
||||
// Only the Vault path of the consumer entry is exposed as an output.
|
||||
|
||||
resource "random_bytes" "munge_key" {
|
||||
length = 128
|
||||
}
|
||||
|
||||
resource "tls_private_key" "admin" {
|
||||
algorithm = "ED25519"
|
||||
}
|
||||
|
||||
// Regenerated whenever the rotation marker changes; the special set avoids
|
||||
// shell quoting issues in the password application command.
|
||||
resource "random_password" "consumer" {
|
||||
length = 24
|
||||
special = true
|
||||
override_special = "-_=+"
|
||||
|
||||
keepers = {
|
||||
rotation = var.credentials_version
|
||||
}
|
||||
}
|
||||
|
||||
// Applies the consumer password on the head node, on day 0 and on every
|
||||
// rotation. The password never travels through user data, so a rotation
|
||||
// never touches the instances: this resource is replaced (triggered by the
|
||||
// rotation marker), re-runs chpasswd over SSH with the management key, and
|
||||
// the Vault entry below is updated in the same apply.
|
||||
// null_resource instead of terraform_data: the tofu-controller runner ships a
|
||||
// Terraform older than 1.4, which has no terraform_data built-in.
|
||||
resource "null_resource" "consumer_password" {
|
||||
// Re-applied on rotation AND whenever the head node is recreated: a fresh
|
||||
// instance boots without the consumer password (it never travels through
|
||||
// user data).
|
||||
triggers = {
|
||||
rotation = var.credentials_version
|
||||
head_node = ovh_cloud_project_instance.head.id
|
||||
}
|
||||
|
||||
connection {
|
||||
type = "ssh"
|
||||
host = local.head_public_ip
|
||||
user = var.admin_username
|
||||
private_key = tls_private_key.admin.private_key_openssh
|
||||
timeout = "15m"
|
||||
}
|
||||
|
||||
provisioner "remote-exec" {
|
||||
inline = [
|
||||
"cloud-init status --wait > /dev/null || true",
|
||||
"echo '${var.username}:${random_password.consumer.result}' | sudo chpasswd",
|
||||
]
|
||||
}
|
||||
|
||||
depends_on = [ovh_cloud_project_instance.head]
|
||||
}
|
||||
|
||||
resource "vault_kv_secret_v2" "consumer" {
|
||||
mount = "secret"
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/${var.username}"
|
||||
data_json = jsonencode({
|
||||
username = var.username
|
||||
password = random_password.consumer.result
|
||||
host = local.head_public_ip
|
||||
port = 22
|
||||
})
|
||||
|
||||
depends_on = [null_resource.consumer_password]
|
||||
}
|
||||
|
||||
resource "vault_kv_secret_v2" "admin" {
|
||||
mount = "secret"
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/admin"
|
||||
data_json = jsonencode({
|
||||
username = var.admin_username
|
||||
ssh_private_key = tls_private_key.admin.private_key_openssh
|
||||
ssh_public_key = trimspace(tls_private_key.admin.public_key_openssh)
|
||||
})
|
||||
}
|
||||
194
hpc/ovh/main.tf
194
hpc/ovh/main.tf
@ -1,194 +0,0 @@
|
||||
// HPC cluster: one head node (the single entry point, public SSH + private
|
||||
// cluster network) and N compute nodes (private network only, internet egress
|
||||
// through an OVH gateway for package installation). The cloud provider
|
||||
// credentials and the project (OVH_CLOUD_PROJECT_SERVICE) reach the provider
|
||||
// from the runner environment, never from module inputs.
|
||||
|
||||
// The flavors/images data sources require an explicit service_name (unlike
|
||||
// the resources, they do not fall back to the OVH_CLOUD_PROJECT_SERVICE
|
||||
// runner variable when the attribute is an explicit null), so the project is
|
||||
// resolved from the credential itself when no override is given: the
|
||||
// per-account API keys of the platform are scoped to a single project.
|
||||
data "ovh_cloud_projects" "all" {
|
||||
count = var.project_id == null ? 1 : 0
|
||||
}
|
||||
|
||||
locals {
|
||||
service_name = var.project_id != null ? var.project_id : one(data.ovh_cloud_projects.all[0].projects[*].project_id)
|
||||
}
|
||||
|
||||
data "ovh_cloud_project_flavors" "head" {
|
||||
service_name = local.service_name
|
||||
name_filter = var.head_flavor
|
||||
region = var.region
|
||||
}
|
||||
|
||||
data "ovh_cloud_project_flavors" "compute" {
|
||||
service_name = local.service_name
|
||||
name_filter = var.compute_flavor
|
||||
region = var.region
|
||||
}
|
||||
|
||||
data "ovh_cloud_project_images" "all" {
|
||||
service_name = local.service_name
|
||||
region = var.region
|
||||
}
|
||||
|
||||
locals {
|
||||
selected_image = one([for m in data.ovh_cloud_project_images.all.images : m if m.name == var.image])
|
||||
selected_head_flavor = one(data.ovh_cloud_project_flavors.head.flavors)
|
||||
selected_compute_flavor = one(data.ovh_cloud_project_flavors.compute.flavors)
|
||||
|
||||
head_hostname = "${var.name_prefix}-head-${var.deployment_id}"
|
||||
head_ip = cidrhost(var.cluster_cidr, 50)
|
||||
|
||||
compute_hostnames = [for i in range(var.compute_node_count) : "${var.name_prefix}-compute${i + 1}-${var.deployment_id}"]
|
||||
compute_ips = [for i in range(var.compute_node_count) : cidrhost(var.cluster_cidr, 60 + i)]
|
||||
}
|
||||
|
||||
// Private cluster network (vRack) with one subnet. Fixed addresses are
|
||||
// assigned outside the DHCP pool; DHCP stays enabled so the instances
|
||||
// configure their private interface from the OpenStack metadata.
|
||||
resource "ovh_cloud_project_network_private" "cluster" {
|
||||
service_name = local.service_name
|
||||
name = "${var.name_prefix}-hpc-${var.deployment_id}"
|
||||
regions = [var.region]
|
||||
vlan_id = var.vlan_id
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_network_private_subnet" "cluster" {
|
||||
service_name = local.service_name
|
||||
network_id = ovh_cloud_project_network_private.cluster.id
|
||||
region = var.region
|
||||
network = var.cluster_cidr
|
||||
// The allocation pool must cover the fixed addresses: the OVH API refuses
|
||||
// an instance ip outside it ("instance ip must be in subnet allocation
|
||||
// pool"). The DHCP agent ports allocate from the start of the pool, so the
|
||||
// fixed addresses (head at 50, compute nodes from 60) stay clear of them:
|
||||
// an instance on a DHCP-port address is silently canceled by the cloud.
|
||||
start = cidrhost(var.cluster_cidr, 10)
|
||||
end = cidrhost(var.cluster_cidr, 200)
|
||||
dhcp = true
|
||||
}
|
||||
|
||||
locals {
|
||||
openstack_network_id = one([
|
||||
for r in ovh_cloud_project_network_private.cluster.regions_attributes : r.openstackid if r.region == var.region
|
||||
])
|
||||
}
|
||||
|
||||
// Internet egress for the whole cluster: the gateway attaches to the
|
||||
// cluster subnet and DHCP distributes it as the default route. A NAT
|
||||
// through a dual-homed head node is not an option: the cloud's port
|
||||
// anti-spoofing silently drops the forwarded return traffic.
|
||||
resource "ovh_cloud_project_gateway" "egress" {
|
||||
service_name = local.service_name
|
||||
name = "${var.name_prefix}-hpc-gw-${var.deployment_id}"
|
||||
model = var.gateway_model
|
||||
region = var.region
|
||||
network_id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
|
||||
// One project SSH key per deployment, managed by the module so it is removed
|
||||
// on destroy. The instance level ssh_key_create is avoided on purpose: keys it
|
||||
// creates outlive the instance and collide on the next deployment.
|
||||
resource "ovh_cloud_project_ssh_key" "admin" {
|
||||
service_name = local.service_name
|
||||
name = "${var.name_prefix}-${var.deployment_id}-key"
|
||||
public_key = trimspace(tls_private_key.admin.public_key_openssh)
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_instance" "head" {
|
||||
service_name = local.service_name
|
||||
region = var.region
|
||||
billing_period = "hourly"
|
||||
name = local.head_hostname
|
||||
|
||||
boot_from {
|
||||
image_id = local.selected_image.id
|
||||
}
|
||||
|
||||
flavor {
|
||||
flavor_id = local.selected_head_flavor.id
|
||||
}
|
||||
|
||||
ssh_key {
|
||||
name = ovh_cloud_project_ssh_key.admin.name
|
||||
}
|
||||
|
||||
// The head node cannot sit on the public network: the cloud refuses an
|
||||
// instance attached to both the public network and a subnet served by a
|
||||
// gateway. Public reachability comes from a floating ip routed through
|
||||
// the gateway instead. The fixed address stays clear of the DHCP agent
|
||||
// ports at the start of the pool: an instance on a DHCP-port address is
|
||||
// silently canceled by the cloud, which also broke the floating ip
|
||||
// creation until the address moved.
|
||||
network {
|
||||
private {
|
||||
ip = local.head_ip
|
||||
floating_ip_create {
|
||||
description = "${var.name_prefix}-hpc-head-${var.deployment_id}"
|
||||
}
|
||||
network {
|
||||
id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
user_data = data.cloudinit_config.head.rendered
|
||||
|
||||
depends_on = [ovh_cloud_project_gateway.egress]
|
||||
}
|
||||
|
||||
// Resolves the public (floating) address of the head node: the instance
|
||||
// addresses only carry the fixed private ip, the floating ip is matched
|
||||
// through the port it is associated with.
|
||||
data "ovh_cloud_project_floatingips" "all" {
|
||||
service_name = local.service_name
|
||||
region_name = var.region
|
||||
|
||||
depends_on = [ovh_cloud_project_instance.head]
|
||||
}
|
||||
|
||||
resource "ovh_cloud_project_instance" "compute" {
|
||||
count = var.compute_node_count
|
||||
service_name = local.service_name
|
||||
region = var.region
|
||||
billing_period = "hourly"
|
||||
name = local.compute_hostnames[count.index]
|
||||
|
||||
boot_from {
|
||||
image_id = local.selected_image.id
|
||||
}
|
||||
|
||||
flavor {
|
||||
flavor_id = local.selected_compute_flavor.id
|
||||
}
|
||||
|
||||
ssh_key {
|
||||
name = ovh_cloud_project_ssh_key.admin.name
|
||||
}
|
||||
|
||||
network {
|
||||
private {
|
||||
ip = local.compute_ips[count.index]
|
||||
network {
|
||||
id = local.openstack_network_id
|
||||
subnet_id = ovh_cloud_project_network_private_subnet.cluster.id
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
user_data = data.cloudinit_config.compute[count.index].rendered
|
||||
|
||||
depends_on = [ovh_cloud_project_gateway.egress]
|
||||
}
|
||||
|
||||
locals {
|
||||
head_public_ip = one([
|
||||
for f in data.ovh_cloud_project_floatingips.all.cloud_project_floatingips : f.ip
|
||||
if try(f.associated_entity.ip, "") == local.head_ip
|
||||
])
|
||||
}
|
||||
@ -1,28 +0,0 @@
|
||||
// Deliberately restricted to the entry point: the response payload returned
|
||||
// to the backend must carry strictly the head node access information, never
|
||||
// the compute node details.
|
||||
|
||||
output "head_node_ip" {
|
||||
description = "Public IPv4 address of the head node, the single entry point of the cluster"
|
||||
value = local.head_public_ip
|
||||
}
|
||||
|
||||
output "head_node_hostname" {
|
||||
description = "Hostname of the head node"
|
||||
value = local.head_hostname
|
||||
}
|
||||
|
||||
output "ssh_port" {
|
||||
description = "SSH port of the head node"
|
||||
value = 22
|
||||
}
|
||||
|
||||
output "vmIps" {
|
||||
description = "Entry point address list, kept for compatibility with the single VM notification flow"
|
||||
value = [local.head_public_ip]
|
||||
}
|
||||
|
||||
output "credentials_ref" {
|
||||
description = "Vault path of the generated consumer credentials for the head node"
|
||||
value = "secret/data/${vault_kv_secret_v2.consumer.name}"
|
||||
}
|
||||
@ -1,102 +0,0 @@
|
||||
variable "project_id" {
|
||||
description = "OVH public cloud project id (service_name). Defaults to the OVH_CLOUD_PROJECT_SERVICE variable of the runner environment, derived from the cloud account configuration."
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
description = "OVH region for all cluster instances and the private network, for example SBG5 or GRA11"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "deployment_id" {
|
||||
description = "Unique deployment identifier, used to build resource names"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "cloud_account" {
|
||||
description = "Cloud account (environment) name, second segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "requester_unique_id" {
|
||||
description = "Requester unique id, third segment of the Vault credentials path"
|
||||
type = string
|
||||
}
|
||||
|
||||
variable "name_prefix" {
|
||||
description = "Prefix for all resource names"
|
||||
type = string
|
||||
default = "simpl"
|
||||
}
|
||||
|
||||
variable "username" {
|
||||
description = "Consumer username on the head node, last segment of the Vault credentials path. The user exists on every node with the same uid (Slurm runs jobs under it), but can only log in on the head node."
|
||||
type = string
|
||||
default = "simpl_user"
|
||||
}
|
||||
|
||||
variable "admin_username" {
|
||||
description = "Management username created on all nodes with SSH key access only"
|
||||
type = string
|
||||
default = "simpladmin"
|
||||
}
|
||||
|
||||
variable "credentials_version" {
|
||||
description = "Opaque rotation marker for the consumer credential. Changing the value regenerates the consumer password, applies it on the head node over SSH and republishes the fresh credential to Vault in the same apply, without touching the cluster nodes. Managed by the credential renewal workflow, not by the deployment script author."
|
||||
type = string
|
||||
default = "1"
|
||||
}
|
||||
|
||||
variable "image" {
|
||||
description = "Image name for every cluster node, must be a cloud-init enabled image"
|
||||
type = string
|
||||
default = "Ubuntu 24.04"
|
||||
}
|
||||
|
||||
variable "head_flavor" {
|
||||
description = "Instance flavor for the head node, for example b3-8"
|
||||
type = string
|
||||
default = "b3-8"
|
||||
}
|
||||
|
||||
variable "compute_flavor" {
|
||||
description = "Instance flavor for the compute nodes, for example b3-8"
|
||||
type = string
|
||||
default = "b3-8"
|
||||
}
|
||||
|
||||
variable "compute_node_count" {
|
||||
description = "Number of compute nodes in the cluster"
|
||||
type = number
|
||||
default = 2
|
||||
|
||||
validation {
|
||||
condition = var.compute_node_count >= 1
|
||||
error_message = "The cluster needs at least one compute node."
|
||||
}
|
||||
}
|
||||
|
||||
variable "compute_cpus" {
|
||||
description = "Number of schedulable CPUs per compute node, declared to the Slurm controller. Must not exceed the vCPUs of compute_flavor."
|
||||
type = number
|
||||
default = 2
|
||||
}
|
||||
|
||||
variable "cluster_cidr" {
|
||||
description = "CIDR of the private cluster network. The head node takes the 50th host address, compute nodes take the 60th onward (lower pool addresses are left to the DHCP agent ports)."
|
||||
type = string
|
||||
default = "192.168.100.0/24"
|
||||
}
|
||||
|
||||
variable "vlan_id" {
|
||||
description = "vRack VLAN id for the private cluster network, must be unused in the project"
|
||||
type = number
|
||||
default = 212
|
||||
}
|
||||
|
||||
variable "gateway_model" {
|
||||
description = "Model (size) of the cloud gateway providing internet egress to the cluster"
|
||||
type = string
|
||||
default = "s"
|
||||
}
|
||||
@ -1,28 +0,0 @@
|
||||
terraform {
|
||||
required_providers {
|
||||
ovh = {
|
||||
source = "ovh/ovh"
|
||||
version = "2.3.0"
|
||||
}
|
||||
vault = {
|
||||
source = "hashicorp/vault"
|
||||
version = "4.8.0"
|
||||
}
|
||||
random = {
|
||||
source = "hashicorp/random"
|
||||
version = "3.7.2"
|
||||
}
|
||||
tls = {
|
||||
source = "hashicorp/tls"
|
||||
version = "4.0.6"
|
||||
}
|
||||
null = {
|
||||
source = "hashicorp/null"
|
||||
version = "3.2.4"
|
||||
}
|
||||
cloudinit = {
|
||||
source = "hashicorp/cloudinit"
|
||||
version = "2.3.7"
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -12,18 +12,17 @@ Provisions a managed Kubernetes cluster with one node pool and generates scoped
|
||||
| `cloud_account` | Cloud account (environment) name, second segment of the Vault credentials path, filled by the provisioner from the cloudAccount header via the `{CLOUD_ACCOUNT}` placeholder |
|
||||
| `requester_unique_id` | Requester unique id, third segment of the Vault credentials path, filled by the provisioner from the requesterUniqueId header via the `{REQUESTER_UNIQUE_ID}` placeholder |
|
||||
| `username` | Consumer ServiceAccount name, defaults to `simpl-consumer`, last segment of the Vault credentials path |
|
||||
| `credentials_version` | Opaque rotation marker for the consumer credential, defaults to `1`. Changing the value recreates the consumer ServiceAccount and its token Secret, revoking every previously issued consumer token and publishing a fresh credential to Vault. Managed by the credential renewal workflow, not by the deployment script author |
|
||||
| `admin_username` | Management (renewer) ServiceAccount name, defaults to `simpl-renewer` |
|
||||
| `name_prefix` | Prefix for resource names, defaults to `simpl` |
|
||||
| `region` | OVH region for the cluster |
|
||||
| `project_id` | Optional override of the OVH project; by default the provider resolves it from the `OVH_CLOUD_PROJECT_SERVICE` variable of the runner environment, which the provisioner derives from the `OVH_PROJECT_ID` key of the cloud account configuration |
|
||||
| `project_id`, `region` | OVH project and region |
|
||||
| `k8s_version`, `k8s_update_policy` | Cluster version and upgrade policy |
|
||||
| `nodepool_*` | Node pool sizing: flavor, desired/min/max nodes, autoscale, anti affinity, billing |
|
||||
| `private_network_id`, `nodes_subnet_id`, `gateway_ip` | Optional vRack attachment, all null for a public cluster |
|
||||
| `token_ttl_seconds` | TTL of the consumer ServiceAccount token, defaults to 3600 |
|
||||
|
||||
Cloud credentials are never module inputs, they reach the provider from the runner environment.
|
||||
|
||||
Credentials, following the existing Vault contract: the OVH admin kubeconfig (client certificates) is never delivered. A `kubernetes` provider chained on the fresh cluster creates, in the same apply, the consumer and the management ServiceAccounts (cluster-admin), each with a non expiring legacy token Secret: credentials stay valid until a renewal is requested, and the renewal revokes them by changing the `credentials_version` input, which makes Terraform delete and recreate the ServiceAccount and its token Secret in the next apply. Both are delivered as assembled kubeconfigs written by the module to Vault: `secret/{provider}/{cloud_account}/{requester_unique_id}/{username}` with the consumer kubeconfig (companions: token, server, ca_data) and the `admin` sibling entry with the root kubeconfig and the management username. No secret material is ever part of an output or of the user body. The Vault provider configuration is generated by the provisioner next to the deployment script, and it authenticates through the Vault Kubernetes auth method with the service account of the Terraform runner, against a Vault role configured at installation time.
|
||||
Credentials, following the existing Vault contract: the OVH admin kubeconfig (client certificates) is never delivered. A `kubernetes` provider chained on the fresh cluster creates, in the same apply, the consumer ServiceAccount (cluster-admin, bound token with TTL) and the management ServiceAccount (cluster-admin, non expiring legacy token Secret, the credential the renewal workflow uses on day 2 and revocable by deleting the ServiceAccount). Both are delivered as assembled kubeconfigs written by the module to Vault: `secret/{provider}/{cloud_account}/{requester_unique_id}/{username}` with the consumer kubeconfig (companions: token, server, ca_data, ttl_seconds) and the `admin` sibling entry with the root kubeconfig and the management username. No secret material is ever part of an output or of the user body. The Vault provider configuration is generated by the provisioner next to the deployment script, and its token comes from the runner environment.
|
||||
|
||||
Note: this module configures the `kubernetes` provider internally (chained on the cluster it creates), so the module block must not use `count`, `for_each` or `depends_on`.
|
||||
|
||||
@ -45,6 +44,7 @@ The provisioner rewrites the `{KUBERNETES_SOURCE}` placeholder to this module's
|
||||
module "kubernetes" {
|
||||
source = "{KUBERNETES_SOURCE}"
|
||||
deployment_id = "{UUID}"
|
||||
project_id = "<ovh project id>"
|
||||
region = "<region>"
|
||||
cloud_account = "{CLOUD_ACCOUNT}"
|
||||
requester_unique_id = "{REQUESTER_UNIQUE_ID}"
|
||||
|
||||
@ -1,10 +1,8 @@
|
||||
// Consumer and management credentials are generated here, never injected by
|
||||
// the backend and never delivered through outputs. The OVH admin kubeconfig
|
||||
// (client certificates) is never delivered either: consumer and management
|
||||
// each get a scoped ServiceAccount with a NON EXPIRING legacy token, valid
|
||||
// until a credential renewal is requested (the renewal deletes and recreates
|
||||
// the ServiceAccount, which revokes every previously issued token at once).
|
||||
// Vault entries follow the existing contract:
|
||||
// (client certificates) is never delivered either: the consumer receives a
|
||||
// scoped ServiceAccount token with a TTL, day 2 renewal uses a dedicated
|
||||
// root token. Vault entries follow the existing contract:
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/{username} consumer kubeconfig
|
||||
// secret/{provider}/{cloud_account}/{requester_unique_id}/admin root kubeconfig + admin username
|
||||
// Only the Vault path of the consumer entry is exposed as an output.
|
||||
@ -17,30 +15,14 @@ resource "kubernetes_namespace_v1" "access" {
|
||||
depends_on = [ovh_cloud_project_kube_nodepool.this]
|
||||
}
|
||||
|
||||
// Rotation anchor: replacing this resource (any change to credentials_version)
|
||||
// cascades to the consumer ServiceAccount and its token Secret through
|
||||
// replace_triggered_by, performing the credential rotation inside Terraform.
|
||||
// null_resource instead of terraform_data: the Terraform of the tofu
|
||||
// controller runner (1.3.x) predates the terraform_data resource type.
|
||||
resource "null_resource" "credentials_version" {
|
||||
triggers = {
|
||||
version = var.credentials_version
|
||||
}
|
||||
}
|
||||
|
||||
// Consumer access: ServiceAccount + non expiring legacy token Secret. The
|
||||
// binding references the ServiceAccount by name, so it survives the
|
||||
// delete/recreate performed by the credential rotation. Recreating the
|
||||
// ServiceAccount changes its UID, which revokes every token issued for it.
|
||||
// Consumer access: ServiceAccount + bound token with TTL. The binding
|
||||
// references the ServiceAccount by name, so it survives the delete/recreate
|
||||
// performed by the credential renewal workflow.
|
||||
resource "kubernetes_service_account_v1" "consumer" {
|
||||
metadata {
|
||||
name = var.username
|
||||
namespace = kubernetes_namespace_v1.access.metadata[0].name
|
||||
}
|
||||
|
||||
lifecycle {
|
||||
replace_triggered_by = [null_resource.credentials_version]
|
||||
}
|
||||
}
|
||||
|
||||
resource "kubernetes_cluster_role_binding_v1" "consumer_admin" {
|
||||
@ -61,20 +43,14 @@ resource "kubernetes_cluster_role_binding_v1" "consumer_admin" {
|
||||
}
|
||||
}
|
||||
|
||||
resource "kubernetes_secret_v1" "consumer_token" {
|
||||
resource "kubernetes_token_request_v1" "consumer" {
|
||||
metadata {
|
||||
name = "${var.username}-token"
|
||||
name = kubernetes_service_account_v1.consumer.metadata[0].name
|
||||
namespace = kubernetes_namespace_v1.access.metadata[0].name
|
||||
annotations = {
|
||||
"kubernetes.io/service-account.name" = kubernetes_service_account_v1.consumer.metadata[0].name
|
||||
}
|
||||
}
|
||||
|
||||
type = "kubernetes.io/service-account-token"
|
||||
wait_for_service_account_token = true
|
||||
|
||||
lifecycle {
|
||||
replace_triggered_by = [null_resource.credentials_version]
|
||||
spec {
|
||||
expiration_seconds = var.token_ttl_seconds
|
||||
}
|
||||
}
|
||||
|
||||
@ -137,7 +113,7 @@ locals {
|
||||
}]
|
||||
users = [{
|
||||
name = var.username
|
||||
user = { token = kubernetes_secret_v1.consumer_token.data["token"] }
|
||||
user = { token = kubernetes_token_request_v1.consumer.token }
|
||||
}]
|
||||
contexts = [{
|
||||
name = "simpl"
|
||||
@ -174,9 +150,10 @@ resource "vault_kv_secret_v2" "consumer" {
|
||||
name = "ovh/${var.cloud_account}/${var.requester_unique_id}/${var.username}"
|
||||
data_json = jsonencode({
|
||||
kubeconfig = local.consumer_kubeconfig
|
||||
token = kubernetes_secret_v1.consumer_token.data["token"]
|
||||
token = kubernetes_token_request_v1.consumer.token
|
||||
server = local.cluster_server
|
||||
ca_data = local.cluster_ca
|
||||
ttl_seconds = var.token_ttl_seconds
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@ -1,7 +1,6 @@
|
||||
variable "project_id" {
|
||||
description = "OVH public cloud project id (service_name). Defaults to the OVH_CLOUD_PROJECT_SERVICE variable of the runner environment, derived from the cloud account configuration."
|
||||
description = "OVH public cloud project id (service_name)"
|
||||
type = string
|
||||
default = null
|
||||
}
|
||||
|
||||
variable "region" {
|
||||
@ -36,18 +35,18 @@ variable "username" {
|
||||
default = "simpl-consumer"
|
||||
}
|
||||
|
||||
variable "credentials_version" {
|
||||
description = "Opaque rotation marker for the consumer credential. Changing the value recreates the consumer ServiceAccount and its token Secret, revoking every previously issued consumer token and publishing a fresh credential to Vault. Managed by the credential renewal workflow, not by the deployment script author."
|
||||
type = string
|
||||
default = "1"
|
||||
}
|
||||
|
||||
variable "admin_username" {
|
||||
description = "Management (renewer) ServiceAccount name, stored in the Vault admin entry"
|
||||
type = string
|
||||
default = "simpl-renewer"
|
||||
}
|
||||
|
||||
variable "token_ttl_seconds" {
|
||||
description = "TTL of the consumer ServiceAccount token (TokenRequest expirationSeconds)"
|
||||
type = number
|
||||
default = 3600
|
||||
}
|
||||
|
||||
variable "k8s_version" {
|
||||
description = "Kubernetes version for the cluster, must be one of the versions currently supported by OVH MKS"
|
||||
type = string
|
||||
|
||||
@ -12,9 +12,5 @@ terraform {
|
||||
source = "hashicorp/vault"
|
||||
version = "4.8.0"
|
||||
}
|
||||
null = {
|
||||
source = "hashicorp/null"
|
||||
version = "3.2.4"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Loading…
Reference in New Issue
Block a user