Gateway API

Every request into this platform — public or private — goes through Gateway API, never a Kubernetes Ingress. Cilium implements the controller (io.cilium/gateway-controller, see Cilium); this page covers the resource model and the three Gateways built on it. Where certificates come from is PKI & Secrets — this page only covers how they attach at the listener.

The resource model

Three roles, three resources:

• GatewayClass — which controller handles a Gateway. Two exist in this platform: cilium, created automatically once gatewayAPI.enabled: true is set in the Cilium Helm values (no manifest for it in-repo), and cilium-tailscale, hand-authored in infrastructure/base/gapi/tailscale-gatewayclass.yaml because it needs a parametersRef to a CiliumGatewayClassConfig. A third class, envoy-ai-gateway, coexists for the opt-in LLM platform’s own data plane and is out of scope here.
• Gateway — listeners, hostnames, and TLS, owned by the platform. Every Gateway lives in the infrastructure namespace. The two Tailscale Gateways — with the GatewayClass and its config, the shared private certificate, and the L7-proxy allow policy — are shared base (infrastructure/base/gapi/); platform-public is per-cluster (infrastructure/aws-0/gapi/, infrastructure/gcp-0/gapi-public/).
• HTTPRoute — routing rules, owned by whatever creates the backend Service. HTTPRoutes attach to a Gateway via parentRefs and only take effect if the Gateway’s allowedRoutes permits the route’s namespace.

The three Gateways

platform-public (infrastructure/aws-0/gapi/platform-public-gateway.yaml) is deliberately narrow, not a general-purpose public entry point. Today its only consumer is runlore’s Slack interactivity callback — Slack posts button clicks from Slack’s own servers, so a private Tailscale gateway can’t carry them. The exposure is bounded by construction: allowedRoutes restricts attachment to the runlore namespace only, the HTTPRoute itself matches exactly one path and method, and every request that reaches the app is HMAC-verified against the Slack signing secret. TLS terminates from cert-manager’s letsencrypt-prod ClusterIssuer (a public CA — this is the one Gateway that does not use OpenBao’s private PKI), and the AWS Load Balancer annotations on infrastructure.annotations make it an internet-facing NLB rather than the Tailscale loadBalancerClass the other two use.

gcp-0 runs its own platform-public (infrastructure/gcp-0/gapi-public/platform-public-gateway.yaml) on the same pattern: one listener per public hostname — today just the cross-cloud probe endpoint — each with its own certificate issued through cert-manager’s gateway-shim annotations, and allowedRoutes pinned to the infrastructure namespace.

allowedRoutes is a namespace allowlist — and a real trap

Both Tailscale Gateways restrict allowedRoutes to a matchExpressions namespace selector, not a wildcard:

# infrastructure/base/gapi/platform-tailscale-general-gateway.yaml
allowedRoutes:
  namespaces:
    from: Selector
    selector:
      matchExpressions:
        - key: kubernetes.io/metadata.name
          operator: In
          values:
            - apps
            - demo
            - envoy-ai-gateway-system
            - envoy-gateway-system
            - observability
            - tooling

Every namespace an App claim can target must be listed here, or its HTTPRoute is rejected NotAllowedByListeners and the App XR never goes Ready — the workload itself runs fine, only the route is refused, so it reads as a broken application rather than a Gateway ACL. This list has to stay in lockstep with wherever claims actually land; there is no automation that keeps them in sync today.

TLS termination

A Gateway listener references a Secret that cert-manager keeps populated — it doesn’t request or manage certificates itself:

listeners:
  - name: https
    port: 443
    protocol: HTTPS
    tls:
      mode: Terminate
      certificateRefs:
        - name: private-gateway-tls

Both Tailscale Gateways share the same wildcard Secret (private-gateway-tls, from the Certificate in infrastructure/base/gapi/platform-private-gateway-certificate.yaml), so when cert-manager renews it, both Gateways pick up the new certificate without a redeploy — no coordination needed between them. See PKI & Secrets for the certificate chain, the ClusterIssuer, and rotation.

ExternalDNS

ExternalDNS watches Gateways and HTTPRoutes (sources: [service, ingress, gateway-httproute] in infrastructure/base/external-dns/helmrelease.yaml) and creates matching DNS records automatically — Route53 on aws-0, no manual DNS step for a new hostname. Two extraArgs keep it scoped correctly:

• --gateway-namespace=infrastructure — only the platform’s own namespace is watched for Gateways.
• --gateway-label-filter=external-dns=enabled — and within it, only Gateways carrying the external-dns: enabled label, which is why every Gateway manifest in this repository sets that label.

policy: sync means ExternalDNS also deletes records when their HTTPRoute is deleted, not just creates them. IAM comes from EKS Pod Identity on aws-0, per the platform constitution — no static AWS credentials.

On gcp-0, two instances run instead of one: infrastructure/gcp-0/external-dns/ overrides the shared release to provider: google against the private Cloud DNS zone (--google-zone-visibility=private), and infrastructure/gcp-0/external-dns-public/ is a second release that assumes the AWS Route53 role over federated web identity to manage the public zone — see ADR-0019.

Routing rules

Beyond a plain PathPrefix match, HTTPRoute supports header matching and weighted backendRefs for canary-style traffic splits — standard Gateway API capability, not something this repository extends. Nothing here is platform-specific enough to be worth its own example; see the Gateway API HTTPRoute reference for the full matching and weighting syntax.

Troubleshooting

Gateway stuck Waiting for controller / GatewayClass ACCEPTED=Unknown — almost always the Cilium operator’s one-shot Gateway API CRD probe; see Cilium.

HTTPRoute has no status.parents, or shows NotAllowedByListeners — the route’s namespace isn’t in the parent Gateway’s allowedRoutes selector (see above), or the hostname doesn’t match a listener’s hostname pattern.

503 upstream connect error ... connection timeout from a service behind a Gateway — the Cilium Gateway API L7 proxy (Envoy) connects to backend pods using the reserved:ingress identity, which a standard CiliumNetworkPolicy cannot select via podSelector/namespaceSelector. A namespace with its own default-deny policy (flux-system, runlore today) needs an explicit fromEntities: [ingress] allow — infrastructure/base/gapi/allow-gateway-l7-proxy.yaml is that policy, scoped by namespace rather than endpointSelector: {} so it doesn’t also open up unrelated pods like Kyverno’s admission webhook. See Policies for the default-deny model this works around.

Certificate not issued / Gateway has no TLS — see PKI & Secrets.

Related

• Cilium — the controller implementing all of this.
• Private Access — how the two Tailscale Gateways enforce access control.
• PKI & Secrets — where the certificates these listeners reference come from.