PREQUEL-2025-0002
Envoy metrics scraping failure with unexpected EOFMediumImpact: 7/10Mitigation: 3/10
Description
Prometheus is failing to scrape and write Envoy metrics from Istio sidecars due to an unexpected EOF error. This occurs when trying to collect metrics from services that don't have proper protocol selection configured in their Kubernetes Service definition\n
Mitigation
Add the appProtocol: http field to the port definition in the Kubernetes Service:\n\n```yaml\napiVersion: v1\nkind: Service\nmetadata:\n name: some-service\nspec:\n type: ClusterIP\n ports:\n - name: web\n port: 9898\n protocol: TCP\n targetPort: 9898\n appProtocol: http\n```\n\n- Alternatively, ensure port naming follows Istio convention (http-*, http2-*, grpc-*, etc.)\n- Restart affected services after configuration changes\n- Verify Prometheus is properly configured with tls_config for scraping Istio-enabled services\n
References
- https://github.com/prometheus/prometheus/issues/10213
- https://github.com/prometheus/prometheus/issues/10213#issuecomment-1064280828
- https://istio.io/latest/docs/ops/configuration/traffic-management/protocol-selection/
- https://kubernetes.io/docs/concepts/services-networking/service/#application-protocol