Reading the Spool

The Processor's job ends when the file is renamed into place. Everything after that belongs to the consumer: noticing the file, processing it, and removing it.

The consumer can be anything that can read a folder. A shell script, a cron job, a systemd path unit, a log shipper's file input, or a watcher inside your own application. This page gives the rules every consumer must follow, and some working examples.

The three rules

1. Only touch files that do not end in the temporary suffix. The suffix is .tmp by default. A .tmp file is still being written. A file with its final name is complete. There is no in-between state, because the rename happens in one step.

2. Delete or move each file when you have processed it. The Processor never removes files. If you set a folder size limit (PG_RELAY_SPOOL_MAX_BYTES), every file counts towards it, temporary files included. When the folder reaches the limit, the Processor stops writing until you make room. The events wait in the queue, but nothing moves until you delete something.

3. Expect each file name to appear once, unless the channel uses replace: true. With replace on, a file can be overwritten at any time. Read it in one go. Do not assume it is still the same file a moment later.

And one practical rule: process the backlog when you start. Files that arrived while your consumer was not running are sitting in the folder. A watcher that only reacts to new files will miss them.

A shell consumer using inotifywait

inotifywait comes from the inotify-tools package, which every major Linux distribution has. It watches a folder and prints a line when something happens.

Watch for moved_to. That is the rename step, which is the exact moment a file becomes complete.

#!/bin/bash
# Consume the 'exports' channel: process each finished file, then delete it.
set -euo pipefail
SPOOL=/var/spool/pg_relay/exports

process_one() {
    local f=$1
    # ... do the work here: load it, send it on, whatever the job is ...
    rm -- "$f"
}

# First, the backlog: anything that arrived before we started.
shopt -s nullglob
for f in "$SPOOL"/*.json; do process_one "$f"; done

# Then, live: every file renamed into the folder from now on.
inotifywait -m -q -e moved_to --format '%f' "$SPOOL" | while read -r name; do
    case "$name" in
        *.tmp) continue ;;          # still being written, ignore it
    esac
    process_one "$SPOOL/$name"
done

Run this as a systemd service under the consumer's own user. That user should be in the spool group from Setting Up the Spool Directory. Set Restart=always. The backlog loop at the top means a restart never misses a file.

A polling consumer

Sometimes inotifywait is not an option. A network filesystem, a container without the tools, or Windows. In those cases, check the folder every few seconds instead. Sorting by name gives time order if your template starts with {queued_at}.

#!/bin/bash
set -euo pipefail
SPOOL=/var/spool/pg_relay/exports
shopt -s nullglob
while true; do
    for f in $(ls -1 "$SPOOL"/*.json 2>/dev/null | sort); do
        process_one "$f"
    done
    sleep 5
done

The same idea in PowerShell, for a Windows Processor server:

$spool = 'D:\spool\pg_relay\exports'
while ($true) {
    Get-ChildItem -Path $spool -File -Filter '*.json' | Sort-Object Name | ForEach-Object {
        # ... process $_.FullName ...
        Remove-Item -LiteralPath $_.FullName
    }
    Start-Sleep -Seconds 5
}

-Filter '*.json' already leaves out .tmp files. If your channel uses a different extension, filter on that one.

Reading a replace: true channel

When a file holds the current state of something, the consumer usually does not delete it. It reads the latest version whenever it needs to. Two things keep that safe:

• Read the whole file in one call. Use cat, Get-Content -Raw, or a single read in your code. Do not hold the file open while you do other work. If the Processor replaces the file while you have the old one open, you keep reading the old, complete version. Your next open sees the new one. You never see a mix.
• Do not watch for close_write. The Processor writes and closes the temporary file. The final name only ever appears through a rename. Watch for moved_to, or poll.

If old files on a replace channel should be cleaned up one day, for example when an order is closed, that is still the consumer's job. The Processor never deletes anything.

When the folder fills up

If the consumer falls behind and the folder reaches its limit, the Processor holds back new files for that channel. The events stay in the queue. Nothing is lost. The Processor logs one spool_unavailable warning, which names the limit and the folder.

In the database you can see the events piling up:

SELECT channel, pending, oldest_pending
FROM pgrelay.queue_stats()
WHERE action_type = 'pg_relay.file_spool';

Watch pending and oldest_pending. That is the signal to alert on.

As soon as the consumer deletes enough files, the next write goes through and the Processor logs spool_available. A held-back event keeps its queue row, so its {event_id} is the same when it finally lands.

---

Next: Configuration and Failure Reference.