PREQUEL-2025-0085
Kafka message size limit exceededMediumImpact: 7/10Mitigation: 5/10
Description
The Kafka producer encountered a \"Message size too large\" error when attempting to send a message to a Kafka broker.\nThis occurs when a message exceeds the configured maximum message size limit on the broker. Kafka has configurable\nmessage size limits at both broker and producer levels to protect system stability and prevent resource exhaustion.\nWhen this limit is hit, the message is rejected and not stored in the topic.\n
Mitigation
- Increase the message size limits if appropriate for your use case:\n On the broker: modify 'message.max.bytes' in server.properties\n On the topic: set 'max.message.bytes' for specific topics with high volume needs\n On the producer: increase 'max.request.size' to match broker settings\n- Implement message chunking in your application to break large payloads into smaller messages\n- Enable and optimize compression (gzip, snappy, lz4) to reduce message size\n- Review data models and serialization formats to minimize payload size\n- Add monitoring for message sizes to detect trends before hitting limits\n- Consider alternative communication patterns for very large data transfers (direct file transfers, object storage)\n