How to configure istio for mesh federation without service discovery?

Question

Multi-trust deployment model from istio documentation

I want to connect multiple meshes together. I currently manage 3 different AKS clusters

• Operations (aks-ops-euwest-1)
• Staging (aks-stg-euwest-1)
• Production (aks-prod-euwest-1)

I have Hashicorp Vault running on Operations, I’d like to be able to reach eg. Postgres that’s running in Staging and Production using istio mTLS (for automatic secret rotation).

Each of the clusters are running istio (Multi-Primary) in different networks. Each cluster has a different ClusterName, MeshID, TrustDomain and NetworkID. Though, the cacerts secrets are configured with a common Root CA

From the istio documentation, to enable cross-cluster communication, a special eastwestgateway has to be deployed. The tlsMode is AUTO_PASSTHROUGH.

These are the environment variables for the eastwestgateway

# sni-dnat adds the clusters required for AUTO_PASSTHROUGH mode
- name: ISTIO_META_ROUTER_MODE
  value: "sni-dnat"
# traffic through this gateway should be routed inside the network
- name: ISTIO_META_REQUESTED_NETWORK_VIEW
  value: aks-ops-euwest-1

I don’t want to enable automatic service discovery by sharing secrets between clusters. Why? Because I want a fine-grained control over which services are going to be exposed between the meshes. I want to be able to specify AuthorizationPolicies that point to serviceaccounts from the remote clusters (because different trust domains)

Eg:

# production cluster
kind: AuthorizationPolicy
spec:
  selector:
    matchLabels:
      app: postgres
  rules:
  - from:
      source:
        - principal: spiffe://operations-cluster/ns/vault/sa/vault

This is from the istio documentation

In some advanced scenarios, load balancing across clusters may not be desired. For example, in a blue/green deployment, you may deploy different versions of the system to different clusters. In this case, each cluster is effectively operating as an independent mesh. This behavior can be achieved in a couple of ways:

• Do not exchange remote secrets between the clusters. This offers the strongest isolation between the clusters.
• Use VirtualService and DestinationRule to disallow routing between two versions of the services.

What the istio documentation doesn't specify, is how to enable cross-cluster communication in the case where secrets are not shared. When sharing secrets, istiod will create additional envoy configuration that will allow pods to transparently communicate through the eastwestgateway. What it doesn't specify, is how to create those configurations manually when not sharing secrets.

The tlsMode is AUTO_PASSTHROUGH. Looking at the istio repository

    // Similar to the passthrough mode, except servers with this TLS
    // mode do not require an associated VirtualService to map from
    // the SNI value to service in the registry. The destination
    // details such as the service/subset/port are encoded in the
    // SNI value. The proxy will forward to the upstream (Envoy)
    // cluster (a group of endpoints) specified by the SNI
    // value. This server is typically used to provide connectivity
    // between services in disparate L3 networks that otherwise do
    // not have direct connectivity between their respective
    // endpoints. Use of this mode assumes that both the source and
    // the destination are using Istio mTLS to secure traffic.
    // In order for this mode to be enabled, the gateway deployment
    // must be configured with the `ISTIO_META_ROUTER_MODE=sni-dnat`
    // environment variable.

The interesting part is The destination details such as the service/subset/port are encoded in the SNI value.

It seems that when sharing secrets between clusters, istio will add envoy configurations that will effectively encode these service/subset/port into the SNI value of an envoy cluster. Though, when secrets are not shared, how can we acheive the same result?

I've looked at this repository, but it is outdated and does not make use of the eastwestgateway.

I've also posted questions on the istio forum here and here, but it's difficult to get help from there.