GitOps

OpenShift GitOps (Argo CD) is the sole operating model for cluster state in this design: operators, HyperConverged, the grid UDN, and namespaces, plus every workload — Symphony management, the Windows compute pool, installer-repo, the image pipeline, KEDA — all reconcile from git. An app-of-apps root Application per cluster is applied once by scripts/bootstrap-gitops.sh; everything after that point is Argo sync, not oc apply.

Purpose

bootstrap-gitops.sh installs the openshift-gitops-operator Subscription, grants the Argo application-controller service account cluster-admin (needed to manage VM/operator resources cluster-wide), creates the repo-grid repository credential Secret, and applies a single root Application named sym-aws-root pointing at gitops/clusters/rosa. That directory name predates D9 (self-hosted IPI replaced ROSA HCP) and is kept as-is to minimize churn — it now means "the AWS-side cluster," not the ROSA platform. The roks bootstrap directory does not exist yet; it lands with Phase 3.

Bootstrap and secrets are irreducibly imperative: nothing can be GitOps-managed before GitOps itself exists, and secrets never enter git by design. scripts/create-secrets.sh is the single entry point for everything that must not be committed — admin passwords, master IPs, the entitlement file — reading them from env vars or a gitignored local file, rendering templates through sed, and piping straight to oc apply. No secret value is ever written to a file this repo tracks.

Placement

Property Value

Property	Value
Operator	`openshift-gitops-operator`, namespace `openshift-operators`, channel `latest`, `installPlanApproval: Automatic`
Argo CD instance	`openshift-gitops` namespace (Red Hat’s default install namespace)
Root Application	`sym-aws-root` in `openshift-gitops`, source path `gitops/clusters/rosa`, `targetRevision: main`, destination `https://kubernetes.default.svc` (in-cluster only)
Sync policy	`automated: { prune: true, selfHeal: true }`, `CreateNamespace=true` — drift is corrected automatically, not just detected
Repo credential	HTTPS + PAT, not SSH. SSH hit a source-IP restriction on the git host that rejected every cluster NAT egress IP; HTTPS+PAT was the durable fix, not an IP allowlist chase
Repo layout precedent	Red Hat’s Cloud Experts guide for VMs on ROSA blesses managing `VirtualMachine` manifests via Argo CD over a Kustomize base/overlay split — the same shape this repo’s `gitops/apps/` + `gitops/clusters/<cluster>/` split follows, using one root Application per cluster rather than an `ApplicationSet`

Operator

openshift-gitops-operator, namespace openshift-operators, channel latest, installPlanApproval: Automatic

Argo CD instance

openshift-gitops namespace (Red Hat’s default install namespace)

Root Application

sym-aws-root in openshift-gitops, source path gitops/clusters/rosa, targetRevision: main, destination https://kubernetes.default.svc (in-cluster only)

Sync policy

automated: { prune: true, selfHeal: true }, CreateNamespace=true — drift is corrected automatically, not just detected

Repo credential

HTTPS + PAT, not SSH. SSH hit a source-IP restriction on the git host that rejected every cluster NAT egress IP; HTTPS+PAT was the durable fix, not an IP allowlist chase

Repo layout precedent

Red Hat’s Cloud Experts guide for VMs on ROSA blesses managing VirtualMachine manifests via Argo CD over a Kustomize base/overlay split — the same shape this repo’s gitops/apps/ + gitops/clusters/<cluster>/ split follows, using one root Application per cluster rather than an ApplicationSet

The PAT itself never enters git — bootstrap-gitops.sh reads it from a local, gitignored file path and passes it straight into oc create secret generic repo-grid, never echoing or logging the value. One hazard worth naming: create-secrets.sh auto-generates a fresh random EGO_ADMIN_PASSWORD if that one env var is not explicitly supplied (every other password var fails closed if unset). Re-running the script to fix one unrelated ConfigMap, without re-supplying the original value, would silently rotate the already-joined masters' EGO admin credential. The safe pattern, learned in build: pull the existing password back out of the live Secret and pass it through unchanged.

Interfaces

Peer Direction Port / Protocol Purpose

Peer	Direction	Port / Protocol	Purpose
Git host (HTTPS+PAT)	outbound from Argo repo-server	443 HTTPS	Poll/fetch git source for all Applications
Kubernetes API (`kubernetes.default.svc`)	in-cluster	443 HTTPS	Argo application-controller reconciles/applies manifests, `cluster-admin` scoped
OpenShift Marketplace	in-cluster	— (OLM catalog)	Source for the `openshift-gitops-operator` and every operator Subscription Argo manages
`scripts/create-secrets.sh`	out-of-band, imperative	`oc create secret`/`oc apply`	Populates the Secrets/ConfigMaps GitOps-managed workloads consume but never define in git
Argo CD web/API	inbound (operator console)	443 HTTPS (Route, default install)	Human sync-status/diff inspection — not part of the reconciliation path

Git host (HTTPS+PAT)

outbound from Argo repo-server

443 HTTPS

Poll/fetch git source for all Applications

Kubernetes API (kubernetes.default.svc)

in-cluster

443 HTTPS

Argo application-controller reconciles/applies manifests, cluster-admin scoped

OpenShift Marketplace

in-cluster

— (OLM catalog)

Source for the openshift-gitops-operator and every operator Subscription Argo manages

scripts/create-secrets.sh

out-of-band, imperative

oc create secret/oc apply

Populates the Secrets/ConfigMaps GitOps-managed workloads consume but never define in git

Argo CD web/API

inbound (operator console)

443 HTTPS (Route, default install)

Human sync-status/diff inspection — not part of the reconciliation path

Maturity

OpenShift GitOps is a Red Hat-shipped operator (redhat-operators catalog source) — supported product, not custom code.
This is not a Validated Pattern, and the repo says so explicitly (ADR 0007): Validated Patterns' Maintained tier requires "use only supported components," and this design has exactly one component that fails that bar by design — the Host Factory KubeVirt provider (detail). The honest framing is "GitOps-deployable reference architecture with one customer-owned, unsupported-by-Red-Hat extension," borrowing Validated Patterns' maturity vocabulary without claiming the status.
The stale-ConfigMap trap. A code fix committed to firstboot-symphony.ps1 does nothing until create-secrets.sh re-renders the win-sysprep-join ConfigMap from that source and the live cluster content is verified — multiple sessions in this project cost hours because a fix was in git but the render step was never re-run. Re-render-and-verify before every reclone is now a standing discipline, not optional.
Argo’s application-controller runs with cluster-admin, granted explicitly by the bootstrap script — required because it manages HyperConverged, VirtualMachinePool, operator Subscriptions, and namespace creation across the cluster, not just app-namespaced resources. Broader than a typical namespaced Argo install, and a deliberate trade-off for the app-of-apps + cluster-config-in-git model.
gitops/bootstrap/roks/ does not exist yet — Phase 3 adds the second cluster’s app-of-apps when the burst site is built.

Verify

# 1. Confirm the GitOps operator and Argo CD server are healthy
oc get csv -n openshift-operators | grep openshift-gitops
oc rollout status deploy/openshift-gitops-server -n openshift-gitops

# 2. Confirm the repo credential exists and is labeled correctly (never prints the PAT value)
oc get secret repo-grid -n openshift-gitops \
  -o jsonpath='{.metadata.labels.argocd\.argoproj\.io/secret-type}{"\n"}'

# 3. Confirm the root Application and its managed children are Synced/Healthy
oc get applications -n openshift-gitops
argocd app get sym-aws-root --grpc-web

# 4. Confirm no drift: a synced app should show zero OutOfSync resources
argocd app diff sym-aws-root --grpc-web

Continue to KEDA metrics shim, the demo-scoped autoscaler this same reconciliation model deploys in Phase 4.