Building a Metric Document¶

This chapter is for the developer writing the producer — the SQL that decides what to measure and hands the measurement to pg_relay. It walks through assembling the JSON document a pgrelay.notify() call sends on a pg_relay.otlp channel, step by step, with Grafana Alloy as the destination — and, just as importantly, which decisions belong in the channel's configuration (set once by whoever registers the channel) versus in the payload (set by your code, per point).

The Overview explains what pg_relay.otlp is and the Reference lists every field; this page is the how, in the order you'll actually make the decisions.

The two halves of a metric¶

Every point Alloy receives from pg_relay is assembled from two sources, and getting the split right is most of the work:

	Where it's set	Who sets it	What it describes
Resource identity	The channel config (`register()`'s `p_action` JSON)	The DBA or operator, once	The emitter: which service, in which namespace and environment, on which host. The same for every point the channel sends
The point	The `notify()` payload	Your producer code, per point	The measurement: its name, its value, when it was taken, and the dimensions that distinguish this reading from others

Grafana treats the two differently. Resource identity is how it finds and groups the metric — the job label, the environment filter, the service inventory. Data-point attributes are the labels you query by — database="prod", shard="3". Put a resource fact (the environment) into the payload's attributes and it becomes a series label rather than service identity, so the service shows up ungrouped; put a dimension (the database name) into the channel config and every point carries the same value, which is useless as a label. Decide which half each fact belongs to before you write anything.

Step 1 — Decide what you are measuring¶

pg_relay.otlp sends a gauge: a value read at an instant that can go up or down — a count of active connections, a queue depth, a replication lag in seconds, a percentage, an amount of money outstanding. Ask two questions before proceeding:

Is it a point-in-time reading? If the natural shape is a running total that only ever increases (bytes written since startup, requests served) or a distribution (request latencies), that's a counter or histogram, which pg_relay does not yet send (Gauges only, for now). Often you can still express what you want as a gauge — the current rate, or the count in the last interval computed by your query — but be deliberate about it.
Is it one number? One notify() call carries one metric with one value. If you want to report five things, that's five calls (a loop in your function — see Step 8), not one payload with five values.

Step 2 — Name the metric¶

name is the string Grafana will show, so follow the conventions Grafana users expect rather than inventing your own:

snake_case, letters, digits, and underscores: pg_connections_active, not PG Connections (active). Prometheus-style backends will sanitise anything else, and the sanitised name is what you'll have to type in every query.
A stable prefix per domain: everything from your database under pg_, everything from the order pipeline under orders_. Prefixes make the metric browser navigable.
The unit as a suffix, where there is one: _seconds, _bytes, _total (for a count), _ratio for 0–1 fractions. replication_lag_seconds tells the reader the unit; replication_lag makes them guess.
Describe the thing, not the query: pg_connections_active, not count_from_pg_stat_activity.

Don't put the environment, the host, or the database name into the metric name — those are labels, and belong in Steps 3 and 4.

Step 3 — Set the resource identity (channel config, once)¶

This is the half that lives in the channel's configuration. If someone else registers channels, hand them this section; if that's you, register the channel with the three recommended keys set:

SELECT pgrelay.register('otel_metrics',
    '{"endpoint": "http://alloy.internal:4318/v1/metrics",
      "service_name": "pg-collector",
      "service_namespace": "platform",
      "deployment_environment": "_env:DEPLOY_ENV"}',
    p_action_type := 'pg_relay.otlp');

What each one does at the Grafana end:

Config key	Becomes	Why you want it
`service_name`	`service.name`	The service's identity. With no namespace, this is the `job` label
`service_namespace`	`service.namespace`	Groups related services. Grafana derives `job` as `platform/pg-collector`
`deployment_environment`	`deployment.environment.name`	Files the service under `prod`, `staging`, or `dev`. Without it, Grafana Cloud's Application Observability shows the service ungrouped and its baselining does not work

Three practical rules:

Don't use / in service_name or service_namespace — it's the job label's separator, and a slash inside either half breaks the split.
Use _env: for the environment. "_env:DEPLOY_ENV" resolves from the Processor host's environment at send time, so the same channel definition promotes cleanly from staging to production — the database never hard-codes prod. Any resource field accepts _env:, exactly like auth.secret.
Anything else goes in resource_attributes — service.version, service.instance.id, cloud.region, a team tag. It's a flat object typed like data-point attributes; the four keys above (plus host.name, which is always the Processor's hostname and can't be set) are rejected inside it, so a stray entry can never override the real thing.

Register one channel per receiver and per resource identity: if two databases on the same host should appear as two services, that's two channels.

Step 4 — Choose the point's attributes (payload, per point)¶

attributes are the dimensions that distinguish one reading of this metric from another: which database, which shard, which queue, which tenant tier. Two rules matter far more than any formatting detail:

Keep cardinality low. Every distinct combination of attribute values is a separate time series the backend must store. database (a handful of values) is a good label; session_pid or a customer id (thousands, constantly changing) will bury your metrics store. If you're unsure whether a value belongs in a label, it probably doesn't — put it in a log instead.
Use the right JSON type, and it maps itself. A JSON string becomes an OTLP string, a number with no fractional part becomes an integer, a fractional number a double, true/false a boolean (Attribute typing). Nested objects and arrays are rejected with an error naming the key — flatten them into separate attributes.

"attributes": {"database": "prod", "shard_id": 3, "is_replica": false}

The resource facts from Step 3 do not go here — no environment, no host. They're already on the resource.

Step 5 — Take the timestamp when you take the measurement¶

timestamp is required, in RFC 3339, and it is the moment the measurement was taken — not now, not send time. Compute it in the same statement as the value:

to_char(clock_timestamp(), 'YYYY-MM-DD"T"HH24:MI:SS.MSOF')

OF renders the session's UTC offset (+10:00), which is what RFC 3339 requires; .MS keeps millisecond precision. Use clock_timestamp() rather than now() if your function does meaningful work before reaching the metric — now() is frozen at the start of the transaction. Why pg_relay insists on this rather than stamping the send time itself is explained under Why timestamp is required: a queued row can wait a moment before dispatch, and a send-time stamp would silently misplace the reading.

Step 6 — Assemble the document with `jsonb_build_object`¶

Don't concatenate JSON by hand — quoting, escaping, and a stray trailing comma will all bite you eventually. Build it with jsonb_build_object, which produces correct JSON for any value, and cast to text for notify():

SELECT jsonb_build_object(
    'name',       'pg_connections_active',
    'value',      (SELECT count(*) FROM pg_stat_activity WHERE state = 'active'),
    'timestamp',  to_char(clock_timestamp(), 'YYYY-MM-DD"T"HH24:MI:SS.MSOF'),
    'attributes', jsonb_build_object('database', current_database())
)::text;

That yields, for example:

{"name": "pg_connections_active", "value": 42,
 "timestamp": "2026-09-02T10:00:00.123+10:00",
 "attributes": {"database": "prod"}}

Check it against the contract before sending: name a non-empty string; value a JSON number (not a quoted string — jsonb_build_object keeps a count(*) numeric, but a to_char'd value would become a string and be rejected); timestamp present and RFC 3339; attributes flat.

Step 7 — Send it, and verify at each hop¶

SELECT pgrelay.notify('otel_metrics', <the document from Step 6>);

notify() returns the queue row's id. Within about a second the Processor claims it, wraps it in the OTLP envelope, and POSTs it. Verify in order — each hop tells you something different:

pg_relay's audit log — did the send succeed?
```
SELECT status, error, elapsed_ms FROM pgrelay.log
 WHERE channel = 'otel_metrics' ORDER BY actioned_at DESC LIMIT 1;
```
ok means Alloy's receiver answered 2xx. An error with a 4xx and a message naming a field means the document or channel config was rejected before or by the receiver — fix and resend; pg_relay never retries those. retry_scheduled means Alloy was unreachable; it will be retried on the normal retry policy.
Alloy — is the point in the pipeline? Alloy's Prometheus exporter publishes what it has received on its scrape endpoint (http://alloy.internal:12345/metrics by default); look for pg_connections_active.
Grafana — does it look right? Query pg_connections_active{database="prod"}. The resource identity from Step 3 arrives as the series' job (platform/pg-collector) and, depending on your exporter's promotion settings, as resource labels; in Grafana Cloud's Application Observability the service should now be listed under platform in the prod environment. If it's there but ungrouped, revisit Step 3 — the resource keys aren't reaching Grafana.

Step 8 — Make it recurring¶

A single notify() is one point, once. For a metric you sample continuously, wrap Step 6 in a function and schedule it with Scheduled Jobs:

CREATE FUNCTION report_pg_metrics() RETURNS void LANGUAGE plpgsql AS $$
DECLARE
    ts text := to_char(clock_timestamp(), 'YYYY-MM-DD"T"HH24:MI:SS.MSOF');
    m  record;
BEGIN
    -- One notify() per metric: name, value, and this point's own dimensions.
    FOR m IN
        SELECT 'pg_connections_active' AS name,
               (SELECT count(*) FROM pg_stat_activity WHERE state = 'active')::numeric AS value
        UNION ALL
        SELECT 'pg_replication_lag_seconds',
               coalesce(extract(epoch FROM now() - pg_last_xact_replay_timestamp()), 0)
        UNION ALL
        SELECT 'pg_database_size_bytes',
               pg_database_size(current_database())
    LOOP
        PERFORM pgrelay.notify('otel_metrics', jsonb_build_object(
            'name',       m.name,
            'value',      m.value,
            'timestamp',  ts,
            'attributes', jsonb_build_object('database', current_database())
        )::text);
    END LOOP;
END $$;

SELECT pgrelay.schedule_job('report_pg_metrics', '30 seconds', 'SELECT report_pg_metrics()');

Every 30 seconds the job enqueues three points sharing one measurement timestamp, each an independently retryable send. Because value is numeric in every branch, jsonb_build_object emits a JSON number each time; the ::numeric cast on the first branch keeps UNION ALL from coercing the column to text.

Checklist¶

Before you call the producer done:

[ ] Metric is a gauge — a point-in-time reading, one number per notify().
[ ] name is snake_case, prefixed, with its unit as a suffix; no environment/host/database baked in.
[ ] Channel config carries service_name, service_namespace, deployment_environment — no / in the first two; environment via _env:.
[ ] Payload attributes are dimensions only, low-cardinality, correctly typed, flat.
[ ] timestamp is the measurement time, RFC 3339 with an offset, taken in the same statement as the value.
[ ] Document built with jsonb_build_object, value a JSON number.
[ ] Verified at all three hops: pgrelay.log → Alloy → Grafana, and the service appears grouped under the right namespace and environment.

Back to the Configuration and Payload Reference for every field's exact contract, or to Wiring Up Grafana Alloy for the receiver side.