The Split-Brain Risk¶
Please read this chapter before running active-active in production
This is the one failure mode that conservative adoption cannot make fully safe.
A sustained network partition — where part of your cluster genuinely cannot talk to the rest for an extended period — is the scenario conservative adoption cannot make safe, and pg_relay does not self-fence (it does not forcibly shut itself down to avoid this risk) — it keeps both sides of a partition available and working. During a sustained partition, an isolated-but-still-running node keeps processing its own rows, while the majority side also processes them:
- In
multi-nodemode, the duplicate work is limited to the isolated node's own rows. - In
leadermode, the blast radius is larger: the isolated old leader and the majority side's newly elected leader each process the entire eligible backlog, so all pending, pre-partition work can potentially run twice.
This is an accepted, deliberately documented trade-off — pg_relay chooses availability over strict consistency here. If your actions cannot tolerate a rare duplicate run during a sustained partition, either design them to be partition-tolerant (idempotent, or guarded by a business-level unique key, as covered in the User Guide's discussion of notify()), or simply do not run those particular channels active-active. This risk is inherent to asynchronous multi-master systems that have side effects outside the database — it is not something specific to pg_relay.
Continue to Failure and Latency at a Glance for a summary table of every scenario covered in this book.