Skip to main content

CRE-2025-0079

SpiceDB Database Corruption: Critical Table LossCritical
Impact: 10/10
Mitigation: 3/10

CRE-2025-0079View on GitHub

Description

Detects catastrophic SpiceDB database corruption where critical core tables

like `alembic_version` and `relation_tuple_transaction` are missing or dropped.

This represents complete database corruption that renders SpiceDB unable to

perform any authorization operations, causing total permission system failure.


Cause

  • Database corruption from storage failures or disk issues
  • Accidental or malicious table drops during maintenance
  • Failed database restore operations leaving incomplete schema
  • Kubernetes storage volume corruption during pod restarts
  • Manual database modifications that remove core SpiceDB tables
  • Incomplete backup restoration missing critical tables
  • Database migration rollback failures

Mitigation

IMMEDIATE ACTIONS:

  • Stop all SpiceDB services immediately to prevent further damage
  • Isolate the corrupted database to prevent data loss
  • Check PostgreSQL logs for corruption indicators or storage errors

ASSESSMENT:

  • Run: `psql -U spicedb -d spicedb -c "\\dt"` to list existing tables
  • Verify critical tables exist: `alembic_version`, `relation_tuple_transaction`, `metadata`
  • Check for recent backup availability and integrity

RECOVERY:

  • Restore database from most recent known-good backup
  • If no backup available, reinitialize database with migrations:

`spicedb migrate head --datastore-conn-uri=<uri>`

  • Verify data integrity after restoration
  • Test permission operations before returning to production

PREVENTION:

  • Implement automated database backups with integrity verification
  • Monitor database table existence in health checks
  • Use database-level permissions to prevent accidental table drops
  • Implement storage monitoring for early corruption detection

References