Filenames and Replacing Files¶

The channel's action column can hold a small JSON object. It describes the file: its name, the suffix used while writing, its permissions, and what to do if a file with that name already exists. It never says where the file goes. That is set by the environment. See Setting Up the Spool Directory.

SELECT pgrelay.register('exports',
    '{"filename": "{channel}-{event_id}.json",
      "extension_tmp": ".tmp",
      "mode": "0640",
      "replace": false}',
    p_action_type := 'pg_relay.file_spool');

Option	Default	What it does
`filename`	`{channel}-{event_id}.json`	The file name template. See below.
`extension_tmp`	`.tmp`	The suffix added while the file is being written. Consumers must ignore files ending in it. Letters, digits, `.`, `_`, and `-` only. Cannot be empty.
`mode`	`0640`	File permissions after writing, as octal digits. Write it as a string, `"0640"`, or as a number, `640`. Linux and macOS only. On Windows the file gets the folder's permissions.
`replace`	`false`	What to do when a file with this name already exists. See Replacing an existing file.

Every option is optional. An empty action means all four defaults.

If an option is wrong, every event on the channel fails and is not retried. The reason is written to pgrelay.log.error. Fix the channel with update() and send the events again. Examples of wrong options:

A template token that does not exist.
An empty extension_tmp.
A mode that is not octal digits.
A replace that is not true or false.

File name templates¶

The filename option is a template. It can use four tokens:

Token	Becomes	Example
`{channel}`	The channel name, as stored.	`exports`
`{event_id}`	The event's queue id. Unique. Stays the same if the event is delivered again.	`5827131938406400`
`{queued_at}`	When the event was queued, in UTC.	`20260907T041200Z`
`{payload.<key>}`	A top-level value from the payload, if the payload is a JSON object. The value must be a string or a number.	`{payload.order_id}` becomes `4471`

Everything outside a token is copied as is. So you choose the extension and the separators.

Template	Example result	Use it for
`{channel}-{event_id}.json` (the default)	`exports-5827131938406400.json`	One file per event. Always unique. Works with any payload.
`{queued_at}-{event_id}.csv`	`20260907T041200Z-5827131938406400.csv`	Consumers that sort files by name to process them in time order.
`order-{payload.order_id}.json`	`order-4471.json`	One file per order. Add `replace: true` to keep it up to date.
`{payload.customer_id}/…`	not allowed	`/` is a folder separator. The spool folder has no subfolders.

The safe name rule¶

The finished file name must pass these checks:

Only letters, digits, ., _, and -.
No .. anywhere in the name.
Not empty.
Shorter than 200 bytes.

If a name fails any check, the event fails and is not retried. The reason goes to pgrelay.log. No file is written, not even a temporary one. The name itself is not written to the log, because it may contain payload data.

Common causes:

A / or a space in a payload value.
A template that contains ...
A {payload.<key>} where the key is missing from the payload.
A {payload.<key>} where the value is null, true, false, an object, or a list.

Two things to know before you use {payload.<key>}:

The producer controls the file name. If the producer sends "order_id": "12 34", the event fails. Clean the value before you call notify(), or use {event_id}, which pg_relay controls.
The channel name must pass the same checks when you use the default template. A channel called order events makes an unsafe name. Call it order_events instead.

A stable key from the producer, such as an order id, gives the same file name every time that event is delivered. That is what makes redelivery safe. See below.

Replacing an existing file¶

By default, replace is false. If a file with the same name already exists, the Processor leaves it alone. It still reports the event as a success. The audit row says ok, with the note file already exists; write skipped (redelivery of an already-written event) in the error column. The Processor's own log has a spool_exists line.

This is the right behaviour for a redelivered event. If the Processor crashed after renaming the file but before recording success, the event is delivered again. The second attempt finds the file and moves on. It does not disturb a file the consumer may already be reading.

Set "replace": true when the file should always hold the latest state of something, and every event should overwrite the previous version:

-- One file per order. It always holds the latest status.
SELECT pgrelay.register('order_state',
    '{"filename": "order-{payload.order_id}.json", "replace": true}',
    p_action_type := 'pg_relay.file_spool');

SELECT pgrelay.notify('order_state', '{"order_id": 4471, "status": "packed"}');
-- ... later ...
SELECT pgrelay.notify('order_state', '{"order_id": 4471, "status": "shipped"}');

After the second event, order-4471.json holds the shipped version.

The new content is written to the temporary file first. Then the rename swaps the old file for the new one in a single step. A consumer sees either the old file or the new file. It never sees a mix of the two, and there is never a moment with no file at all. The audit row says ok with the note file already existed; replaced (replace=true). The Processor's log has a spool_replaced line.

The first write for a name, when no file exists yet, is a plain ok with no note. That is the same with either setting.

If two events for the same order are waiting at the same time, they are written in queue order. If you use more than one worker or more than one Processor, and you must be certain they never overlap, register the channel with p_concurrency_mode := 'channel_payload'. That serialises events with the same payload. Use 'channel' to serialise the whole channel. This works the same as for any other action type. See Managing Channels.

Overriding `replace` for one event¶

If the payload is a JSON object, it can override the channel setting. Add a top-level "replace" key with true or false:

-- The channel default is replace=false. This one event may overwrite.
SELECT pgrelay.notify('exports', '{"replace": true, "order_id": 4471, "status": "shipped"}');

-- The channel default is replace=true. This one event must not overwrite.
SELECT pgrelay.notify('order_state', '{"replace": false, "order_id": 4471, "status": "packed"}');

The rules are kept narrow on purpose, because the Processor otherwise never reads the payload:

The Processor only looks at payloads that are JSON objects, and only at the top-level "replace" key.
The file still gets the exact bytes you sent. The "replace" key is written to the file with everything else. If the consumer must not see it, use the channel option instead.
If "replace" is present but is not true or false, for example the string "yes", the event fails and is not retried. That is a bug in the producer.
A payload that is not a JSON object, such as CSV, XML, plain text, or a JSON list, is not read at all. It uses the channel setting.

What happens on redelivery and retry¶

Situation	`replace: false`	`replace: true`
The Processor crashed after the rename. The event is delivered again.	The file is kept. The event is `ok` with the "skipped" note.	The file is rewritten with the same bytes. The event is `ok` with the "replaced" note.
A disk error happened before the rename. A retry is scheduled.	The retry writes the file. Nothing was written the first time, so nothing is duplicated. With the default template, the retry has a new `{event_id}`, so its name differs from the failed attempt's.	Same.
Two events produce the same `{payload.<key>}` name.	The second is skipped. The first file wins.	The second overwrites. The latest event wins.

The folder size limit (MAX_BYTES, see the reference) counts a replaced file's old size as freed. Replacing a 1 MB file with another 1 MB file does not count as 2 MB.

Next: Reading the Spool.