Skip to content

Operate Internal DNS Cutover Records (HyOps)

Purpose: Manage internal service endpoint records through platform/network/dns-routing without forcing applications to consume raw node IPs.
Owner: Network/platform engineering
Trigger: DR failover, failback, service endpoint change, or internal DNS reconciliation
Impact: Publishes DNS routing intent and optionally updates the authoritative internal DNS service
Severity: P2
Pre-reqs: PowerDNS authority reachable if provider=powerdns-api, runtime vault decrypt works, and required DNS API credentials are present.
Rollback strategy: Re-run the module with desired: primary|secondary as needed, or run destroy with dns_state: absent if the intent object should be removed.

Context

This runbook covers:

  • Module: platform/network/dns-routing
  • Driver: config/ansible
  • Providers:
  • manual-command for custom/external DNS automation
  • powerdns-api for the shared internal PowerDNS authority

This module is the cutover/update layer, not the DNS authority itself. The current recommended DNS model is:

  • Cloudflare for public zones
  • PowerDNS Authoritative for internal/private zones
  • applications use stable FQDNs such as postgres.dev.hyops.internal

Preconditions and safety checks

  • Installed hyops can run from any working directory.
  • If you want to use the shipped examples, set:
export HYOPS_CORE_ROOT="${HYOPS_CORE_ROOT:-$HOME/.hybridops/core/app}"
  • Choose the right env with --env.
  • Have a stable internal zone already delegated/served by PowerDNS.
  • For powerdns-api apply mode, the PowerDNS API key env var must exist and be listed in required_env.

Example secret seed:

hyops secrets set --env dev POWERDNS_API_KEY='...'

Example overlays

  • Minimal intent-only overlay:
  • $HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.min.yml
  • Manual command apply overlay:
  • $HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.apply.yml
  • State-driven inventory overlay:
  • $HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.state.yml
  • PowerDNS API apply overlay:
  • $HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.powerdns.yml

Steps

  1. Preflight the chosen overlay
hyops preflight --env dev --strict \
  --module platform/network/dns-routing \
  --inputs "$HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.powerdns.yml"
  1. Apply the routing change
hyops apply --env dev \
  --module platform/network/dns-routing \
  --inputs "$HYOPS_CORE_ROOT/modules/platform/network/dns-routing/examples/inputs.powerdns.yml"
  1. Verify resulting state
cat "$HOME/.hybridops/envs/dev/state/modules/platform__network__dns-routing/latest.json"

Check:

  • status is ok
  • outputs.dns.provider is correct
  • outputs.dns.desired matches the requested cutover side
  • outputs.dns.targets contains the expected endpoint(s)

PowerDNS notes

When using provider: powerdns-api:

  • zone is the logical DNS zone name
  • powerdns_zone_id may be omitted when it matches zone
  • record_fqdn should be the final service FQDN
  • CNAME records are normalized with a trailing dot automatically

Typical internal DR record:

  • record_fqdn: postgres.dev.hyops.internal
  • primary_targets: [10.12.0.31]
  • secondary_targets: [10.72.16.24]

Destroy / intent removal

To remove the intent file or mark the DNS routing object absent:

cat >/tmp/hyops-dns-routing.absent.yml <<'YAML'
inventory_groups:
  edge_control:
    - name: edge01
      host: 198.51.100.10
connectivity_check: false
dns_state: absent
YAML

hyops destroy --env dev \
  --module platform/network/dns-routing \
  --inputs /tmp/hyops-dns-routing.absent.yml

Verification

Primary evidence:

  • $HOME/.hybridops/envs/<env>/logs/module/platform__network__dns-routing/<run_id>/

Check:

  • ansible.log
  • driver_result.json
  • the state JSON under the module state path

For PowerDNS, also verify the record in the authority itself via API or UI.

References