Skip to main content

CRE-2025-0104

Istio Ambient traffic fails with timed out waiting for workload from xdsMedium
Impact: 8/10
Mitigation: 7/10

CRE-2025-0104View on GitHub

Description

Ztunnel must fetch pod workload info from Istiod over XDS before tunneling.

If it doesn't receive a response within ~5s, it rejects the connection with:

`timed out waiting for workload … from xds`. Intermittent XDS delays may

indicate Istiod overload or misconfiguration (e.g. PILOT_DEBOUNCE_AFTER).


Cause

  • Istiod is under heavy CPU/memory pressure and slow to respond.
  • Network policies or CNI are blocking port 15012 (XDS) between ztunnel and Istiod.
  • PILOT_DEBOUNCE_AFTER or other Envoy debouncing has been increased.

Mitigation

IMMEDIATE:

  • Check Istiod pod resource usage: `kubectl -n istio-system top pods istiod-xxx`
  • Inspect ztunnel logs for repeated XDS timeouts.
  • Ensure port 15012 TCP is open between all nodes and istiod.

RECOVERY:

  • Scale up Istiod deployment or increase resources.
  • Review meshConfig.PILOT_DEBOUNCE_* and revert experimental changes.

References