Seeking Best Practices for Waypoint Proxy Deployment in Large-Scale Ambient Mesh

Background

We're exploring a transition to ambient mesh in our Kubernetes cluster which contains:

• Thousands of pods and services
• Extensive use of VirtualServices for L7 traffic management features

While the L4 security features with ztunnel seem straightforward, I'm looking for community guidance on waypoint proxy deployment topology for handling L7 features.

Current Concerns

In our current sidecar deployment, network paths follow a predictable pattern:

Client → Client-Sidecar → Server-Sidecar → Server (typically 2 nodes communication)

With ambient mesh, the potential path becomes:

Client → Client's ztunnel → Waypoint → Server's ztunnel → Server

The challenge is that these components might span multiple nodes:

Pod A (Node A) → Waypoint (Node B) → Pod B (Node C)

This cross-node traffic pattern seems more complex than our current sidecar model, potentially introducing additional latency and network hops.

Questions for the Community

1. What's the recommended waypoint deployment topology for a large-scale production environment?

2. Should we deploy waypoints per namespace, per service, or using some other logical grouping?

3. What affinity rules have worked well for waypoint deployment to minimize cross-node traffic?

4. How are others handling waypoint scalability for services with varying traffic patterns?

Thanks in advance for your insights.