Skip to content

High Availability — Multiple Processors

Running more than one Processor

You can run any number of Processor programs at the same time against the same database. They share the work automatically, with no configuration needed.

At startup, each Processor claims the lowest free instance slot (1–64) using a session-level advisory lock (pg_try_advisory_lock(19680305, n)) on its coordinator connection. Slots are released automatically when a connection closes, including after a crash. Each Processor stamps its slot number on every log line, so logs from several Processors are easy to tell apart.

How work is partitioned

pgrelay.queue_pending_ids() uses FOR UPDATE SKIP LOCKED. When two Processors run it at the same moment, they each get a disjoint list of IDs — PostgreSQL's own locking engine handles the split automatically. There is no configuration needed, no shared state to manage, and no coordination protocol to reason about. Each Processor independently claims, processes, and logs events.

Even within a single Processor, multiple workers use this same mechanism to avoid duplicating work between themselves.

Channels with a non-default concurrency_mode keep their exclusivity and ordering guarantees across multiple Processors too — candidate selection happens inside the database, so a serialised channel's next event is only offered to someone once its predecessor has finished, whichever instance actually ran it. See Per-channel dispatch concurrency.

How many Processors to run

For most workloads, one Processor with --workers 2 or --workers 4 is enough. Run multiple separate Processor programs when:

  • You want active/active high availability — automatic failover with no restart needed. Run one Processor per application server. If one server goes down, the others simply keep processing, with no operator action required.
  • You need very high throughput — after you have already exhausted the --workers option on a single Processor.

There is no leader election

pg_relay has no concept of a primary or leader Processor (outside of the optional --mode leader setting covered in the Multi-Master Deployment book, which is a different, opt-in feature). By default, every Processor is identical and does the same thing. The database's own row-level locking (FOR UPDATE SKIP LOCKED) is the coordination mechanism, and it is safe and robust by design.

Continue to Standby and Replica Databases for how the Processor behaves during failover.