CREATE A NEW CUSTOMER CLUSTER (AZURE / AWS, TIER 1-3 OR CUSTOM)¶
Provisions a complete d3VIEW HPC customer cluster on a cloud provider. Picks one of three predefined tiers (or a custom configuration) and provisions the appropriate set of VMs (balancer, applications, scheduler, storage, optional database, optional monitor) plus the VNet, NSGs, NICs, and public IP for the balancer. Drives an underlying make stack on the cloud-ops host (CLOUD/bronze-template). Admin-only; requires CLOUD_PROVISIONING_ENABLED=yes and the Azure system SP credentials configured.
When to use¶
Tagged: azure, aws, cloud, customer, hpc, hpcserver, provision, terraform.
Inputs¶
| Label | ID | Type | Default | Required | Description |
|---|---|---|---|---|---|
| Cloud Provider | cloud_provider | select | azure | ✓ | Which cloud to provision into. AWS is shown for discoverability but the worker will throw a ‘not yet supported’ error if selected. Only Azure is implemented today. |
| Customer Name | customer_name | text | — | ✓ | Short identifier for the customer (alphanumeric + dashes). Used in resource tags and as part of the d3VIEW Hpcserver row name. e.g. ‘acmecorp’. |
| FQDN (fully-qualified domain name) | fqdn | text | — | ✓ | The customer’s domain name (e.g. ‘acme.d3view.com’). Resolves to the balancer’s public IP. Used by HAProxy and the d3VIEW login page. |
| Azure Region | region | text | eastus | ✓ | Azure region to provision in (e.g. eastus, westus2, koreacentral). Must have gallery image-versions replicated (Dennis has eastus, westus3, centralus, koreacentral, southindia today). |
| Tier | tier | select | 3 | ✓ | Predefined cluster topology. Tier 3 matches Dennis’s current default and is the safest first-test target. |
| Wait for make stack to complete? | wait_for_make_stack | select | no | When ‘yes’, the worker polls Azure->pollMakeStackStatus to a terminal state before capturing IPs / SP creds and creating the Hpcserver row. When ‘no’, the worker returns immediately after Azure->provisionCustomer kicks the make stack - downstream pollers / cloud_status workers can pick up status later. Default ‘no’ so the workflow stays responsive. | |
| Register HPC server in THIS d3VIEW’s database? (internal testing only) | register_local_hpcserver | select | no | When ‘yes’: writes an Hpcserver row to THIS d3VIEW instance’s database (3092 when running on dev). Lets us submit jobs from this d3VIEW UI to the newly-provisioned scheduler BEFORE the customer’s d3VIEW is fully set up with license + admin account. When ‘no’ (production default): the worker provisions infrastructure only; the customer’s d3VIEW is responsible for registering its own Hpcserver row after install. Set to ‘yes’ for d3view-internal-testing runs only. | |
| Application VM count | application_count | number | 1 | Number of application VMs to provision. Only used when tier=custom (for tier=1 the worker defaults to 2; tier=2/3 always 1). | |
| Enable dedicated database VM | enable_database | select | no | Only used when tier=custom. Tier 1/2 always provision dedicated DB; Tier 3 always embeds. | |
| Enable monitor VM | enable_monitor | select | no | Only used when tier=custom. Tier 1 always provisions monitor; Tier 2/3 never do. | |
| Balancer VM size | balancer_vm_size | select | Standard_F2 | ✓ | Azure VM SKU for the balancer (HAProxy + SSH bastion + only VM with public IP). |
| Balancer OS disk (GB) | balancer_os_disk_gb | number | 30 | ✓ | OS disk size in GB. 30 GB matches Dennis’s default. |
| Application VM size | application_vm_size | select | Standard_F4 | ✓ | Azure VM SKU for the application VM(s). The application VM runs d3VIEW web (Apache + PHP) - and for Tier 3 also MySQL. |
| Application OS disk (GB) | application_os_disk_gb | number | 60 | ✓ | OS disk size. Tier 3 needs more since MySQL data lives here too. |
| Scheduler VM size | scheduler_vm_size | select | Standard_F4 | ✓ | Azure VM SKU for the scheduler VM (Slurm controller + Resume/Suspend hooks). |
| Scheduler OS disk (GB) | scheduler_os_disk_gb | number | 30 | ✓ | |
| Storage VM size | storage_vm_size | select | Standard_F2s_v2 | ✓ | Storage VM exports the shared NFS used by all other VMs. Higher SKU = better throughput. NOTE: today’s internal-prm-d4s submodule hardcodes Standard_D4s_v4. Allowing this input to actually take effect requires a TF parameterization refactor - see DESIGN_NOTES.md Q4 (deferred). |
| Data disk count | storage_data_disk_count | number | 2 | ✓ | Number of data disks attached to the storage VM (Dennis hardcodes 2: data + scratch). Defaults to 2. SCAFFOLD ONLY - honored by TF refactor in v2 (DESIGN_NOTES.md Q4). |
| Data disk size each (GB) | storage_data_disk_size_gb | number | 4096 | ✓ | Per-disk size. Dennis defaults to 4096 (4 TB) for both data + scratch disks. SCAFFOLD ONLY - honored by TF refactor in v2. |
| Data disk tier | storage_data_disk_tier | select | Premium_LRS | ✓ | Disk performance tier. Premium = best IO for solver scratch + working files. SCAFFOLD ONLY - honored by TF refactor in v2. |
| Database VM size | database_vm_size | select | Standard_F4 | Only used for Tier 1, Tier 2, or custom tier with enable_database=yes. Ignored otherwise. SCAFFOLD ONLY - also requires updating application-setup.yml to point d3VIEW at the dedicated DB VM (DESIGN_NOTES.md Q5, deferred to Tier 2 phase). | |
| Database OS disk (GB) | database_os_disk_gb | number | 30 | ||
| Database data disk (GB) | database_data_disk_size_gb | number | 512 | Size of the data disk that holds MySQL data files. Defaults to 512 GB; bump for high-volume customers. | |
| Monitor VM size | monitor_vm_size | select | Standard_F2 | Only used for Tier 1 or custom tier with enable_monitor=yes. Ignored otherwise. | |
| Monitor OS disk (GB) | monitor_os_disk_gb | number | 30 |
Outputs¶
| Label | ID | Type | Description |
|---|---|---|---|
| Customer ID (CID) | cid | text | 12-char hex CID generated by class.Azure.inc::generateCid. Internal-test customers always start with f0xx (cost-attribution prefix). |
| Pool RG | pool | text | Pool resource group the customer was placed in (e.g. d3v-pool-eastus-0). Hosts the shared compute gallery. |
| Customer Directory | customer_dir | text | Path on the cloud-ops host where the customer’s Terraform state lives. Format: /opt/cloud/customer-data/regions/<r>/<pool>/<cid>. |
| Azure Resource Group | resource_group | text | Name of the Azure RG containing all the customer’s resources. Format: d3v-<cid>-rg. |
| Resolved Tier | tier_resolved | text | Echo of the tier the worker resolved this customer to (‘1’ / ‘2’ / ‘3’ / ‘custom’) after applying defaults. |
| Modules Provisioned | modules_provisioned | textarea | Comma-separated list of VM modules actually provisioned (e.g. ‘balancer,application1,scheduler,storage,database,monitor’). |
| Balancer Public IP | balancer_public_ip | text | Public IP of the balancer VM - customer’s FQDN should DNS-resolve here. Empty if make stack didn’t complete. |
| Balancer Internal IP | balancer_internal_ip | text | Balancer’s IP on the customer VNet (typically 192.168.240.254). |
| Application Internal IPs | application_ips | textarea | Comma-separated internal IPs of the application VM(s) (typically 192.168.240.4, .5, .8). |
| Scheduler Internal IP | scheduler_internal_ip | text | Scheduler VM internal IP (typically 192.168.240.7). Used by d3VIEW’s HpcjobLauncher to dispatch sbatch. |
| Storage Internal IP | storage_internal_ip | text | Storage VM internal IP (typically 192.168.240.9). NFS exports /shared. |
| Database Internal IP | database_internal_ip | text | Database VM internal IP (typically 192.168.240.6). Empty when DB is embedded on application (Tier 3). |
| Monitor Internal IP | monitor_internal_ip | text | Monitor VM internal IP (typically 192.168.240.253). Empty when monitor is disabled. |
| Per-Customer SP App ID | sp_app_id | text | The appId of the per-customer Service Principal auto-created by make principal.o. Has Contributor on customer RG + pool RG + template RG only. Used by the scheduler VM’s Resume/Suspend programs. |
| Per-Customer SP Password | sp_password | text | Plaintext password of the per-customer SP - captured ONCE here, otherwise lives only in <customer-dir>/config/principal/principal.json. Treat as a secret; mask in logs / UI display. |
| Per-Customer SP Tenant | sp_tenant | text | Tenant ID for the per-customer SP (same as the d3view.com tenant). |
| Per-Customer SP Display Name | sp_display_name | text | Display name in Azure AD. Format: d3v-<cid>-sp. |
| Hpcserver Row ID | hpcserver_id | text | ID of the d3VIEW Hpcserver row created for this cluster, with ipaddress wired to balancer_public_ip and options.cloud_provisioning_meta populated. |
| Make stack status | make_stack_status | text | One of: not_waited / in_progress / complete / failed / timed_out. Filled only if the worker was configured to block + poll. |
| Make stack Log Path | make_stack_log_path | text | Absolute path to make-stack.log on the cloud-ops host. Tail this for live progress. |
| Execution Log | execution_log | textarea | Structured per-step trace of the worker’s run (Azure->provisionCustomer, Hpcserver row creation, etc.) with durations + outcomes. First place to look on any failure. |
Disciplines¶
- platform.cloud
- platform.hpc
Auto-generated from platform schema. Worker id: cloud_create_customer. Schema hash: 837e90bf7e71. Hand-curated docs in workerexamples/ override this page when present.