Skip to main content

CRE-2025-0099

Redpanda Crash Due to Memory Exhaustion and Startup FailuresHigh
Impact: 8/10
Mitigation: 7/10

CRE-2025-0099View on GitHub

Description

Redpanda streaming platform crashes due to a combination of system-level failures including

permission denied errors for performance monitoring subsystems, missing critical configuration

files, and memory allocation failures.


Cause

Multi-factor failure combining insufficient kernel permissions (perf_event_paranoid settings),

missing persistent configuration files (config_cache.yaml, .bootstrap.yaml), and container

resource limits triggering OOMKiller. The memory_abort_on_alloc_failure setting causes

immediate termination on memory exhaustion.


Mitigation

  1. Configure kernel parameters: echo 1 > /proc/sys/kernel/perf_event_paranoid
  2. Ensure proper volume mounts for /var/lib/redpanda/data and /etc/redpanda directories
  3. Increase container memory limits and implement resource quotas
  4. Verify file system permissions and ownership (redpanda:redpanda)
  5. Implement readiness/liveness probes with appropriate restart policies

References