Processor Health Snapshots¶

The Processor is often the machine you cannot see: a service on a VM, a container, a box in someone else's rack, quietly polling the queue. Health snapshots (v1.3) let you ask it, through the database you both share, to report on the host it is running on, and to leave the answer in a table where you (or a monitoring job) can read it back. Five request types ship: live utilisation (os_metrics), the process table (processes), and three database-host audit types — kernel and tuning posture (os_config), path-to-device storage resolution (storage), and network interfaces (network).

The mechanism follows the same pattern as pg_relay.reload: a reserved action type, an ordinary channel, an ordinary pgrelay.notify(). There is no new protocol — a health request is just a queue event, so scheduling, deferral (p_run_at), expiry (p_expire_at), deduplication, and pgrelay.cancel() all work on it unchanged.

One-time setup¶

Register a channel whose action type is the reserved pg_relay.health. The action itself is empty — the Processor performs the work, not SQL:

SELECT pgrelay.register('sys_health', '', p_action_type := 'pg_relay.health');

Any channel name works (there are no reserved channel names); one such channel per database is typically all you need.

Making a request¶

The payload of the notify() call is the request. All of it is optional:

-- Everything defaulted: os_metrics, no reference, disk = the Processor's working directory
SELECT pgrelay.notify('sys_health');

-- A bare string is shorthand for just the calling reference
SELECT pgrelay.notify('sys_health', 'deploy-42');

-- The full form: a JSON object
SELECT pgrelay.notify('sys_health',
    '{"reference": "deploy-42", "request_type": "os_metrics", "disk": "/var/lib"}');

-- The process table: every process, or only those matching a search string
SELECT pgrelay.notify('sys_health', '{"request_type": "processes"}');
SELECT pgrelay.notify('sys_health',
    '{"request_type": "processes", "search": "postgres", "reference": "incident-4711"}');

-- The database-host audits: kernel/tuning posture, storage resolution for
-- the paths that matter, network interfaces
SELECT pgrelay.notify('sys_health', '{"request_type": "os_config"}');
SELECT pgrelay.notify('sys_health',
    '{"request_type": "storage", "paths": ["/var/lib/pgsql/18/data", "/var/lib/pgsql/18/data/pg_wal"]}');
SELECT pgrelay.notify('sys_health', '{"request_type": "network"}');

Key	Default	Meaning
`reference`	none (NULL)	A caller-chosen correlation string, echoed verbatim onto the answer row — how you find your answer among many.
`request_type`	`os_metrics`	What to collect. v1.3 implements `os_metrics`, `processes`, `os_config`, `storage`, and `network`; the vocabulary is designed to grow (installed versions, …) without schema changes.
`disk`	the Processor's working directory	`os_metrics` only. An OS-specific path qualifying the space calculation: `/var/lib` on Linux, `D:\` on Windows. An unusable path does not fail the request — the reason lands under the answer's `errors.disk` key.
`search`	none (every process)	`processes` only. A case-insensitive substring filter applied to each process's name and full command line — `"postgres"` finds every backend, `"walwriter"` finds one. Absent or empty returns the whole process table.
`paths`	the Processor's working directory	`storage` only. The paths to resolve to their mounts and devices. Pass the paths that actually matter — the database's `data_directory` and its WAL directory — to learn whether they share a device.

Two payload rules worth knowing: a string that starts with { but is not valid JSON resolves the event as a permanent error (a typo'd request must not silently become a reference), and an unrecognised request_type does the same, naming the types this Processor supports.

Reading the answer¶

The Processor that claims the event answers on its next poll (within about a second) by inserting one row into the private pgrelay.processor_health table. Read it back with pgrelay.health_report() (granted via grant_user(), like the other management functions):

SELECT * FROM pgrelay.health_report(p_reference := 'deploy-42');

id           | 84980853314158703
queue_id     | 84980825153601646        -- the id notify() returned
request_type | os_metrics
reference    | deploy-42
instance     | 1                        -- the Processor instance (1-64) that answered
node         | 0                        -- pg_relay.node_id of its database (0 = single-node)
host         | pg18-rocky9              -- the Processor host's OS hostname
os           | linux
data         | {"cpu_percent": 1, "load1": 0.04,
                "mem_total_bytes": 24925671424, "mem_available_bytes": 22999216128,
                "disk_path": "/var/lib",
                "disk_total_bytes": 192044597248, "disk_available_bytes": 20104380416}
collected_at | 2026-08-23 12:03:34+00

health_report(p_limit, p_reference, p_request_type) returns rows newest first; both filters are exact-match and optional. The identity of the responder lives in real columns; the results live entirely in the data JSONB document, keyed by what the request_type produces — new request types add keys, never columns.

What `os_metrics` reports, per platform¶

The Processor collects with the Go standard library only — no agents, no dependencies — which draws the platform lines below:

Key	Linux	macOS	Windows	How
`cpu_percent`	✓	—	✓	Two counter samples 250 ms apart (`/proc/stat` / `GetSystemTimes`). 0–100 across all cores.
`load1`	✓	✓	—	1-minute load average (`/proc/loadavg` / `vm.loadavg` sysctl). Windows has no load-average concept.
`mem_total_bytes`	✓	✓	✓	`/proc/meminfo` / `hw.memsize` / `GlobalMemoryStatusEx`.
`mem_available_bytes`	✓	✓*	✓	Linux uses `MemAvailable` (free + reclaimable cache). *macOS approximates as free pages × page size, which undercounts by ignoring reclaimable pages.
`disk_total_bytes` / `disk_available_bytes`	✓	✓	✓	`statfs` / `GetDiskFreeSpaceEx` on the `disk` path. "Available" is what an unprivileged process could use.
`disk_path`	✓	✓	✓	Always present: the path the space calculation actually used.

True CPU percentage on macOS requires Mach host calls the standard library cannot reach, so macOS reports load1 only — the honest number, clearly labelled.

Collection is best-effort per metric: anything unavailable is simply omitted from data, with the reason recorded under an errors object ({"errors": {"disk": "statfs /no/such/path: no such file or directory"}}). Only a database failure aborts the event.

What `processes` reports¶

The data document carries the request echo and the matched process table:

{ "search": "postgres", "process_count": 13, "truncated": false,
  "processes": [
    { "pid": 1421, "ppid": 1292, "name": "postgres", "cmdline": "postgres: checkpointer",
      "username": "postgres", "state": "S", "exe": "/usr/pgsql-18/bin/postgres",
      "rss_bytes": 46514176, "vsz_bytes": 225804288, "swap_bytes": 0,
      "threads": 1, "nice": 0, "cpu_seconds": 0.56, "cpu_percent": 0,
      "started_at": "2026-08-22T01:02:33Z", "elapsed_seconds": 129894,
      "read_bytes": 49152, "write_bytes": 67506176, "open_fds": 7 }, … ] }

process_count is the full matched total; the processes array is ordered worst-resident-memory-first and capped at 800 entries — when more matched, truncated is true and the 800 largest by RSS are the ones kept. Each cmdline is likewise capped at 2048 bytes (cmdline_truncated flags a cut).
Two fields deserve a special mention for database work. PostgreSQL rewrites its command line, so cmdline is the backend's live status (postgres: alice appdb 10.0.0.5 idle in transaction). On Linux, exe shows (deleted) when a process is still running a binary that has since been replaced on disk — the classic "upgraded but never restarted" tell. state is the other health signal worth watching: D (uninterruptible I/O wait) and Z (zombie) are findings in themselves.
Each Linux process also carries its effective resource limits and pinning. The limits object (nofile_soft/nofile_hard/nproc_soft/nproc_hard, -1 = unlimited) is read from /proc/<pid>/limits — the limits the process actually got, which is what every limits.conf-versus-systemd-override comparison is really asking. cpus_allowed and mems_allowed give the CPU set and NUMA nodes it is bound to. A PostgreSQL backend showing "nofile_soft": 1024 on a busy host is a finding, straight from the answer.
Per-field availability follows the platform, stdlib-only like everything above. Linux is first-class: every field, including an instantaneous cpu_percent from two samples 250 ms apart, with read_bytes/write_bytes/open_fds/exe readable only for the Processor's own user's processes unless it runs as root (unreadable fields are omitted per process, never errors). Windows (Toolhelp32 snapshot) reports pid/ppid/name/threads plus, where the process can be opened, cpu_seconds, started_at, rss_bytes, and vsz_bytes (commit charge) — no command line, so search matches the executable name only. macOS reports identity, memory, state, nice, cpu_seconds, and started_at via /bin/ps (the one supported doorway to a kernel ABI the standard library cannot express).

Command lines can carry secrets

Anything passed to a program as an argument — passwords, tokens, connection strings — is visible in its command line and will appear in a processes answer. pgrelay.processor_health is a private table read through the granted health_report(), which is the right containment: treat process documents as sensitive and do not relay them onward wholesale.

The database-host audits: `os_config`, `storage`, `network`¶

These three exist for one audience: someone auditing the host a database (or its Processor) runs on. They are Linux collectors by nature — the facts they report are Linux database-host concerns read from /proc, /sys, and /etc — so on macOS and Windows they answer honestly with an explanatory errors object rather than a thin pretence of parity. They are cheap, mostly-static snapshots: the natural use is a scheduled request whose answers a consumer diffs against a baseline, so a quietly changed sysctl, a re-enabled THP, or a disabled SELinux surfaces as drift.

`os_config` — kernel and tuning posture¶

A real answer, verbatim:

{ "kernel": "5.14.0-687.24.1.el9_8.x86_64",
  "os_release": "Rocky Linux 9.8 (Blue Onyx)",
  "glibc": "2.34",
  "locale": "LANG=C.UTF-8",
  "thp_enabled": "never", "thp_defrag": "madvise",
  "sysctl": { "vm.swappiness": 60, "vm.dirty_ratio": 20, "vm.dirty_background_ratio": 10,
              "vm.overcommit_memory": 0, "vm.overcommit_ratio": 50,
              "net.core.somaxconn": 4096, "net.ipv4.tcp_keepalive_time": 7200,
              "net.ipv4.tcp_keepalive_intvl": 75, "net.ipv4.tcp_keepalive_probes": 9 },
  "cgroup_version": 2,
  "selinux": "enforcing",
  "numa_nodes": 1,
  "db_clock_offset_ms": 0 }

Field by field:

Kernel release and OS name.
glibc version. A collation-breaking glibc upgrade is the single most notorious silent killer for PostgreSQL hosts. Comparing this value across a primary and its standbys before an OS upgrade is the early warning that the database's own datcollversion mismatch messages arrive too late to give.
OS locale.
Transparent huge pages — the enabled and defrag modes. This is the other classic latency killer; never/madvise is what you want to see on a database host.
The vm.* and net.* sysctls that govern checkpoint/flush behaviour, overcommit, connection backlog, and replication keepalives — a fixed set.
cgroup version.
SELinux mode (enforcing/permissive/disabled) — often silently disabled and forgotten. apparmor appears on hosts that have it.
NUMA node count.
auto_patching — which automatic-patching configurations are present (dnf-automatic, unattended-upgrades). Presence only, deliberately: patch history belongs to patch-management tooling.
db_clock_offset_ms — the answering Processor's clock measured against the database's clock_timestamp() on the very connection that carried the request. It is a relative-drift signal between the two hosts your audit timestamps depend on, not a substitute for real NTP monitoring.

One documented exception to the stdlib-syscalls-only rule: the glibc version comes from executing libc.so.6 itself (it is designed to be run for exactly this), the same class of exception as /bin/ps on macOS.

`storage` — the paths that matter, resolved to their devices¶

Pass the database's own paths and the answer tells you what they actually sit on — this real answer is itself a finding (data and WAL sharing one device):

{ "paths": [
    { "path": "/var/lib/pgsql/18/data",
      "mount_point": "/", "fs_type": "xfs",
      "mount_options": "rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota",
      "device": "/dev/mapper/rocky-root", "disk": "sda",
      "rotational": true, "scheduler": "none", "luks": false },
    { "path": "/var/lib/pgsql/18/data/pg_wal",
      "mount_point": "/", "fs_type": "xfs",
      "mount_options": "rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota",
      "device": "/dev/mapper/rocky-root", "disk": "sda",
      "rotational": true, "scheduler": "none", "luks": false } ] }

Per path:

The owning mount point, filesystem type, and full mount options — noatime present or absent, at a glance.
The mounted device and the whole disk beneath it. Device-mapper stacks are followed down through their slaves, so an LVM volume resolves to the real disk whose queue matters.
rotational and the active I/O scheduler from that disk's queue — the none-on-NVMe versus mq-deadline-on-SSD versus wrong-scheduler-entirely check.
luks — whether any device-mapper layer between the filesystem and the disk is dm-crypt.

Two identical disk values across your data and WAL paths answer the shared-versus-dedicated-spindle question directly. A path that does not exist gets a per-path error instead; a network filesystem reports its mount facts and simply omits the block-device ones. Honest limits: thin provisioning and SAN-versus-local are largely invisible from inside a guest, and this collector does not guess.

`network` — interfaces, MTU, bonding¶

{ "interfaces": [
    { "name": "eth0", "mtu": 9000, "operstate": "up", "speed_mbps": 10000,
      "bond": { "mode": "fault-tolerance (active-backup)", "slaves": ["eth0", "eth1"] } } ] }

Per non-loopback interface: MTU (jumbo frames for replication traffic, visible at a glance), link state, speed where the driver reports one, and — when /proc/net/bonding/<iface> exists — the bonding mode and slave interfaces. The connection-backlog and keepalive sysctls that pair with this live in os_config's sysctl object.

Semantics to keep in mind¶

Exactly one instance answers each request. A health event is a normal queue event, claimed under FOR UPDATE SKIP LOCKED — one claimer. If you run several Processor instances and want a fleet survey, send one request per instance and correlate on reference (the answers' instance/host columns tell you who spoke). You do not control which instance claims a given request.
The metrics describe the Processor host, not the database host. If the Processor runs beside PostgreSQL they coincide; if it runs elsewhere, "disk space" is the Processor machine's disk. For the database server's own disk, ask the database server.
The payload is never logged. Like every pg_relay event, the request (and therefore your reference) appears in pgrelay.queue/pgrelay.log and the answer row — database tables — but never in the Processor's stdout, which carries only the channel and queue id (health_reported: sys_health, id: …).

Housekeeping¶

Answers accumulate one row per request. The general pgrelay.purge() now trims this table by default. Its p_processor_health parameter (default true) applies the same retention rule it applies to the audit log — age with p_hours, keep-the-newest-N with p_keep_quantity — so the one scheduled purge call you already run maintains both tables and returns the combined count. Pass p_processor_health := false to leave answers alone, or use the targeted pgrelay.purge_health(p_hours) (default 168 = 7 days) to give them their own retention:

SELECT pgrelay.purge(p_hours := 720);                             -- audit log AND health answers: 30 days
SELECT pgrelay.purge(p_hours := 720, p_processor_health := false); -- audit log only
SELECT pgrelay.purge_health(24);                                   -- health answers only: keep a day

Both health_report() and purge_health() require an explicit grant (pgrelay.grant_user(role) covers them). The table itself is private, like every pg_relay table; the Processor's own write path, pgrelay._write_health(), is part of the operating set (grant_relay(), checked by preflight() as grant:_write_health) and is granted to the pgrelay role automatically during the 1.3 upgrade.

Processor Health Snapshots¶

One-time setup¶

Making a request¶

Reading the answer¶

What os_metrics reports, per platform¶

What processes reports¶

The database-host audits: os_config, storage, network¶

os_config — kernel and tuning posture¶

storage — the paths that matter, resolved to their devices¶

network — interfaces, MTU, bonding¶