Architecture

One platform, one custom component, nothing repackaged. IBM Spectrum Symphony runs exactly as IBM supports it — full Windows Server compute hosts, RHEL management hosts, the traditional installer, the documented port matrix — on a substrate that clones a compute host from a golden image in minutes. This page is the full topology, the supportability spine, and the honest story of how the platform under it changed mid-build.

Thesis

Modernising a Windows compute grid fails one of two ways: change what the ISV supports (repackage Symphony into containers it was never certified for), or change nothing (lift the same static estate onto new hardware). This architecture does neither. It keeps Symphony’s install, its port matrix, and its Windows Server compute hosts untouched, and changes only the substrate underneath: OpenShift Virtualization on bare-metal workers, reconciled by GitOps, with VM-level elasticity in minutes.

Exactly one custom component exists in the whole design: the Host Factory KubeVirt provider (detail) — a ~5-script plug-in against an interface IBM documents for exactly this purpose. Every other layer is somebody’s supported product.

The supportability spine

Layer Product Support

Cluster

OpenShift 4.20 (self-hosted IPI today; ROSA/ROKS the designed extension)

Red Hat

Virtualization

OpenShift Virtualization (GA)

Red Hat

Windows guests

Windows Server 2019, SVVP-certified OCP Virt guest since 4.8

Red Hat cert + Microsoft

Image pipeline

OpenShift Pipelines + windows-efi-installer

Supported platform (OpenShift Pipelines); windows-efi-installer itself is an upstream KubeVirt community example pipeline, adapted in-repo — not a Red Hat-productized artifact

GitOps

OpenShift GitOps (Argo CD), app-of-apps

Red Hat

Demo autoscale

Custom Metrics Autoscaler (KEDA)

Red Hat

Grid

Symphony 7.3.2 Advanced Edition

IBM

Burst provider

Host Factory KubeVirt provider — the one custom piece

Built in-repo; 5-script interface IBM documents; FINOS/Google prior art

When the grid breaks at 3am there is a vendor to call for every layer but one — and that one is a small plug-in against a documented interface, not a fork of anything.

Prior art — where this breaks new ground

Red Hat publishes a mature OpenShift Virtualization pattern this design reuses directly: the Architecture Guide (cluster/node/storage/network/HA guidance) and the Example Architecture: 1000 Virtual Machines (Dec 2025, v1.0.0), which sizes a bare-metal cluster at 14 nodes total (12 for workload capacity + 2 reserved for HA) for 1000 VMs — one configuration in a table ranging 21-200 VMs/host depending on node size. That table, not a single fixed ratio, is the sizing yardstick this reference architecture benchmarks against as it scales past its current single-metal-node footprint.

What does not exist is a Red Hat FSI HPC/grid case study. Red Hat’s own financial-services virtualization material covers general VM consolidation (a bank replacing thousands of hypervisors) but no named Symphony/quant-risk/grid workload. This reference architecture is establishing that pattern, not following one.

Topology (current build)

Self-hosted OpenShift 4.20 IPI (sym-aws) on AWS, per D9: ROSA HCP was blocked by an AWS Marketplace billing link with no CLI/API workaround, so the platform pivoted to installer-provisioned infrastructure, which needs only AWS credentials and a pull secret. The ROKS burst site is the designed Phase-3 extension — not yet built.

AWS us-east-1 -- sym-aws (self-hosted OpenShift 4.20, IPI)         IBM Cloud us-east -- ROKS burst site
+-----------------------------------------------------+           (Phase 3+ -- DESIGNED, NOT YET BUILT)
| Control plane: 3x m6i.xlarge (masters)               |          +---------------------------------------+
| Workers:       3x m6i.xlarge                         |          | ROKS 4.20+ VPC, manual OCP Virt install|
| virt-metal MachineSet: 1x m5zn.metal (bare metal)     |          | VPC bare-metal workers (bx2d/mx2d)     |
|  -- OpenShift Virtualization (HCO) --                 |          |  -- OpenShift Virtualization --        |
|  +---------------------------------------------------+|          |  +-----------------------------------+ |
|  | Symphony mgmt plane (RHEL 9 VMs)                   ||  egoconfig | Windows compute-host VMs           | |
|  |  sym-master1 (primary) + sym-master2               |<-+--join----+->  provisioned by the CUSTOM        | |
|  |  EGO/SOAM, Host Factory                            ||  EGO/SOAM  |    Host Factory KubeVirt provider,  | |
|  +---------------------------------------------------+|  traffic   |    VNI-routable (no NAT)             | |
|  | Windows Server 2019 compute VMs                    ||            |  +-----------------------------------+ |
|  |  VirtualMachinePool, d1 instancetype               ||            |  Storage: ODF on local NVMe            |
|  +---------------------------------------------------+|            +-------------------┬-------------------+
|  Layer2 UDN . no NAT . base port 7869 . MTU 1500      |                                |
|  Storage: gp3 EBS (CDI StorageProfile override)       |                                |
+------------------------┬------------------------------+                                |
                          |                     IPsec VPN (D3, PoC) / Direct Link (production upgrade)
                          └───────────────────────────────────────────────────────────────┘
GitOps: OpenShift GitOps (Argo CD) app-of-apps -- gitops/bootstrap/ -> gitops/clusters/rosa/ -> gitops/apps/*

Measured: IPI install 44m35s end-to-end; metal Machine provision-to-Ready ~20 min; HyperConverged converge ~5 min; first test VM boot 18s. Baseline running cost at this footprint is ~$5.2/hr (3 masters + 3 workers m6i.xlarge + 1x m5zn.metal); scaling the virt-metal MachineSet to 0 when idle saves ~$4/hr, metal re-provisions in ~20 min.

The gitops/clusters/rosa/ directory name predates D9’s rename to sym-aws and has not been renamed — a cosmetic drift, not a functional one.

The D9 story — why "sym-aws" is self-hosted, not ROSA

The original plan targeted ROSA HCP — Red Hat’s Cloud Experts reference deployment for OCP Virt uses HCP, and it is GA for OpenShift Virtualization. Provisioning hit an AWS Marketplace billing link with no CLI or API workaround: ROSA cluster creation requires accepting a Marketplace subscription that could not be automated or scripted around. Rather than block the PoC on a manual console step outside the GitOps model, the platform pivoted (D9) to installer-provisioned infrastructure (OpenShift IPI), which needs only AWS credentials and a pull secret — no Marketplace dependency at all. The rest of the architecture — layer2 UDN, bare-metal MachineSet, GitOps app-of-apps — carried over unchanged; only the cluster-provisioning path changed. See ADR 0009 for the full decision record.

Design invariants (non-negotiable)

Breaking any of these cost real debugging days in this build — full detail on the network model:

  • Same base port (7869) on every host — masters and compute, either site.

  • No NAT between grid hosts — layer2 UDN, real routable IPs, no masquerade.

  • Identical MTU across sites — pinned to 1500 on masters and Windows guests.

  • SSM ports pinned per symphony/config/sd-ssm-ports.xml, not left OS-random.

  • EGO_DYNAMIC_HOST_WAIT_TIME must be set (ships defined at 60s; never delete the key) or dynamic hosts silently fail to join.

  • Bidirectional 7869-7875 firewall establishment — the master’s LIM dials back to a newly-registered compute host; an inbound-only rule on the compute side leaves it silently unavail even though outbound reachability tests pass clean.

Status

Proven on the live build: self-hosted OCP 4.20 IPI cluster, metal MachineSet, OpenShift Virtualization, layer2 UDN, RHEL Symphony management plane (EGO/SOAM healthy, WAN-tuning fragment applied), the two-stage Windows golden-image pipeline, and the compute VirtualMachinePool cloning and booting. Open at time of writing: the Windows compute-host join is the last mile of Phase 2 — fixed-in-source, not yet independently re-confirmed against a fresh clone. Phases 3-5 (interconnect, ROKS burst, KEDA demo, Host Factory provider) are designed, not built. Full detail: SWOT & sign-off.

Continue to Network model.