Cloud support

The platform runs on two clouds: aws-0 on EKS and gcp-0 on GKE Standard. They are not one abstraction with two backends. They are two implementations of the same three-stage model, sharing every Kubernetes-layer component and diverging exactly where the clouds themselves diverge.

This page is the map of that divergence: what each cloud uses, where the two deliberately meet, and what is still missing on GCP.

Status at a glance

Matching managed services

The Kubernetes layer is identical on both clouds — same Cilium, same Flux, same OpenBao, same VictoriaMetrics, same Gateway API. Everything below is the layer where a cloud’s own service is unavoidable, and what stands in for what.

Compute and networking

Storage

Identity and secrets

DNS and certificates

This is the one place the two clouds are deliberately not symmetric, and the asymmetry is the point.

Private DNS is native on each cloud. Only the public zone is centralised — and only because cloud.ogenki.io is a Route 53 zone this repository does not manage, while Let’s Encrypt must resolve _acme-challenge publicly to issue a certificate. Delegating a subdomain to Cloud DNS or moving the zone were both considered and rejected; what made federation acceptable is that it needs no static AWS credential on GCP — cert-manager and external-dns assume an AWS role using a projected Kubernetes ServiceAccount token, the same identity-by-token model both clouds already use internally. The full argument, including what the dependency costs, is in ADR-0019.

Data services

Neither cloud’s managed database is used. PostgreSQL is CloudNativePG and key/value is Valkey — both self-hosted, both driven by a Crossplane claim, and both now work on either cloud:

SQLInstance was the last claim to reach GCP, and until crossplane-configuration v0.4.0 its GCP Composition was a deliberate dead-end — it failed evaluation rather than composing nothing, so a claim said why instead of hanging Ready=Unknown. That was consistent with ADR-0007: a claim that cannot be honoured should say so at reconcile time.

It is a real implementation now. Both clouds render from the same KCL module, differing in where barman writes (gs:// with googleCredentials.gkeEnvironment rather than s3:// with s3Credentials.inheritFromIAMRole) and in the identity that writes — one GCPWorkloadIdentity, bucket-scoped, in place of four AWS IAM resources. The CloudNativePG operator itself is still deployed only on aws-0 (infrastructure/aws-0/cloudnative-pg/), so a claim on gcp-0 needs that overlay before it can reconcile.

Decisions that shaped the split

Seven records carry the multicloud reasoning. Read in this order they explain why the platform looks the way it does on a second cloud:

The shared layer

Two things belong to neither cloud and are provisioned once:

• opentofu/shared/tailscale — one tailnet, both clusters. Its state stays in S3 precisely because the tailnet is not a GCP resource or an AWS one.
• opentofu/shared/aws-gcp-federation — the AWS IAM OIDC provider that trusts the GKE issuer, which is what makes the DNS row above work.

Known gaps

gcp-0 now runs the same layers as aws-0. Four components are still excluded, and the distinction that matters is why — three of them are not gaps at all.

Excluded by design, not missing

• flux-previews — PR preview environments. Running them on both clusters would double-provision every preview and both would write the same public DNS records. Previews belong to one cluster by nature. (It also hardcodes cluster_name: aws-0 while living in the shared flux/ tree, which is worth fixing regardless.)
• karpenter / karpenter-nodepools — GCP uses Node Auto-Provisioning with ComputeClasses instead (ADR-0006).
• eks-pod-identities — GCPWorkloadIdentity is the counterpart, a different Kind rather than a second Composition (ADR-0002).
• ZITADEL — deliberately a singleton on aws-0, addressed through an identity_provider_url variable so the hosting cloud is a stated value rather than an accident of which overlay includes it.

Genuinely not portable yet

• image-gallery — the only application excluded, and not for a manifest reason. It hardcodes STORAGE_ENDPOINT=s3.eu-west-3.amazonaws.com in its container environment and talks to it through an S3 SDK. Reaching Cloud Storage means either GCS’s S3-compatible XML API with HMAC keys — static credentials this platform avoids wherever a workload identity will do — or a GCS-native client. Both are changes to the application.
• runlore — the SRE agent reads ${domain_name}, a key only aws-0 defines, and drives AWS cloud tools. Flux substitutes an undefined variable to empty, so wiring it would render a hostname with a hole in it rather than fail.
• Harbor’s database has no backups on gcp-0. The claim side is ready — the Composition renders barman’s ObjectStore and a bucket-scoped identity as soon as backup is set. The cluster side is not: the barman plugin ships a CiliumNetworkPolicy whose egress is a toFQDNs allowlist of S3 endpoints, so deploying it would let the plugin start and then silently drop every connection to storage.googleapis.com.

Needs a human, on both clouds

Two ExternalSecrets read from OpenBao and nothing seeds them — the same is true on aws-0, where they were written by hand:

• Harbor’s admin and Valkey passwords, at harbor/admin/password
• Flux’s Slack token, at the OpenBao key observability/flux/slack-app

Until they exist, Harbor waits on its secret and Flux alerts are dropped. Reconciliation itself is unaffected.

None of the above are cloud-abstraction failures. The shared layer reconciles identically on both clouds; what is left is one application with a cloud baked into its code, one policy to port, and secrets to seed.

Open question: one identity provider, or two?

ZITADEL is exposed at auth.${public_domain_name}, and that variable is per-cluster — cloud.ogenki.io on aws-0, gcp.cloud.ogenki.io on gcp-0. So the same manifest would render auth.cloud.ogenki.io on one cloud and auth.gcp.cloud.ogenki.io on the other. There is no DNS collision, which is the reassuring half of the answer.

The unresolved half is what that would mean. Two hostnames is two independent ZITADEL instances: two user directories, two session stores, two sets of OIDC clients, and no federation between them. For an identity provider that is usually the wrong outcome, and it would be reached by default rather than by decision.

Today the question is still open rather than urgent, because ZITADEL cannot run on gcp-0 at all — it declares a SQLInstance, and that claim has no GCP implementation (see the table above). Three models are on the table:

1. One instance on aws-0, with gcp-0 workloads authenticating across the cloud boundary. This mirrors the DNS decision — one authoritative service, reachable from the other cloud — and would be the platform’s second deliberate cross-cloud dependency.
2. Two independent instances, which is what the current naming produces by default. Operationally simple, but the two clusters become separate identity islands.
3. An external identity provider, hosted by neither cloud.

No decision has been recorded yet. When one is, it belongs in an ADR alongside ADR-0019, for the same reason that one exists: a dependency that crosses clouds should be argued for in writing, not absorbed silently.

Adding a third cloud

The mechanics of extending this — which values are cloud-neutral, which are per-cluster, and what a new lane must supply — are in Guides → Add a cloud provider.