Skip to content

Provision On-Prem Ops Runner (HyOps Blueprint)

Purpose: Provision and bootstrap a shared on-prem runner VM on the management network so failback and local platform workflows can run from inside the restored on-prem site.
Owner: Platform engineering / SRE
Trigger: On-prem control-plane bootstrap, failback pipeline preparation, or runner rebuild
Impact: Creates one shared on-prem runner VM and bootstraps the HybridOps runner toolchain on it for failback or local execution
Severity: P2
Pre-reqs: Proxmox and the on-prem management network are reachable, the current on-prem Linux template contract is available or buildable (currently rocky-9), and operators have a real management IP/gateway for the runner VM.
Rollback strategy: Run platform/linux/ops-runner#onprem_ops_runner_bootstrap with runner_state: absent if needed, then destroy platform/onprem/platform-vm#onprem_ops_runner.

Context

Blueprint ref: networking/onprem-ops-runner@v1
Location (example file): ~/.hybridops/envs/<env>/config/blueprints/onprem-ops-runner.yml

Step flow:

  1. core/onprem/template-image (template_key: rocky-9)
  2. platform/onprem/platform-vm#onprem_ops_runner
  3. platform/linux/ops-runner#onprem_ops_runner_bootstrap

This blueprint is the preferred on-prem execution bootstrap for:

  • runner-local failback workflows
  • steady-state on-prem platform maintenance
  • local control-plane work that should not run from an operator workstation

The blueprint uses Proxmox-init bastion auto-detection for first-hop SSH readiness and bootstrap, with the Proxmox jump user set explicitly to root, so operators do not need direct workstation reachability to the on-prem management subnet when Proxmox remains reachable.

Preconditions and safety checks

  1. Validate the shipped blueprint and create an env-scoped overlay:
hyops blueprint init --env dev \
  --ref networking/onprem-ops-runner@v1 \
  --dest-name onprem-ops-runner.yml

  1. Edit the overlay and replace the placeholders:

  2. CHANGE_ME_RUNNER_IP_CIDR

  3. CHANGE_ME_RUNNER_GATEWAY

  4. Validate and preflight:

hyops blueprint validate --ref networking/onprem-ops-runner@v1

hyops blueprint preflight --env dev \
  --file "$HOME/.hybridops/envs/dev/config/blueprints/onprem-ops-runner.yml"

Execute

From a source checkout during development:

HYOPS_CORE_ROOT=/path/to/hybridops-core \
hyops blueprint deploy --env dev \
  --file "$HOME/.hybridops/envs/dev/config/blueprints/onprem-ops-runner.yml" \
  --execute

From a packaged install:

hyops blueprint deploy --env dev \
  --file "$HOME/.hybridops/envs/dev/config/blueprints/onprem-ops-runner.yml" \
  --execute

Runner dispatch preserves HYOPS_ENV for remote jobs, so env-scoped naming stays intact on shared metal. That means physical VM names on Proxmox continue to come out as environment-prefixed names such as dev-pgha-01, rather than collapsing to bare logical keys like pgha-01.

Verify

hyops state show --env dev --module platform/onprem/platform-vm#onprem_ops_runner
hyops state show --env dev --module platform/linux/ops-runner#onprem_ops_runner_bootstrap

Expected:

  • platform/onprem/platform-vm#onprem_ops_runner is status: ok
  • platform/linux/ops-runner#onprem_ops_runner_bootstrap is status: ok
  • one VM named like platform-shared-runner-01
  • HybridOps installed under /opt/hybridops/core

How this fits failback

Use this runner as the execution host for:

That keeps the failback posture consistent with cloud failover:

  • operator workstation is only the dispatcher
  • real execution happens on the target-side runner
  • failback does not depend on ad hoc SSH from a laptop

References