Skip to content

fix(database): pin the PostgreSQL data connection to UTC - #1627

Merged
rmyndharis merged 1 commit into
mainfrom
fix/pg-utc-timestamps
Sep 16, 2026
Merged

rmyndharis merged 1 commit into
mainfrom
fix/pg-utc-timestamps

Conversation

@rmyndharis

Copy link
Copy Markdown
Owner

Every timestamp column on the PostgreSQL data connection is timestamp without time zone, which stores no zone at all, and three writers filled those columns with three different conventions: node-postgres binds a JS Date as the gateway process's local wall time, parses a naive timestamp back the same way, and DEFAULT now(), which fills most @CreateDateColumn/@UpdateDateColumn (TypeORM binds no value for those), writes the PostgreSQL session's zone. On a gateway that does not run in UTC one row therefore carries two conventions at once.

That is what shifts a restored backup by the host's offset, and again on each further restore: the export reads a column back through the local-time parser, writes ISO text with a Z, and the import binds that text so the server drops the zone. The same split is not limited to backups. A retention LessThan(cutoff) binds the cutoff in local time and compares it against createdAt written by the server, so on a host east of UTC the sweep deletes rows up to the offset younger than the window names, silently; the today stats window and cross-zone lease comparisons are wrong for the same reason.

  • src/database/postgres-utc.ts pins the connection in all three directions: parseInputDatesAsUTC for binding, a parser for the scalar timestamp OID (1114) that reads naive text as UTC, and SET TIME ZONE 'UTC' on every pooled connection. The parser is registered through pg's per-client types table, and only for the scalar OID: 1115 (timestamp[]) keeps pg-types' array parser, and the spec asserts the schema holds no array-of-timestamp column.
  • The session pin is a statement on the connection rather than the startup options parameter, which is not guaranteed to survive a pooler and is already spoken for by the search_path of a non-public schema. It is issued through pg-pool's own onConnect hook, which covers every client the pool opens (the first one during DataSource.initialize() included) and owns two things a Client subclass cannot reach. The pool attaches the client's error listener before calling the hook, so a socket that dies mid-statement fails that one acquire instead of reaching Node as an unhandled error event and ending the process; and it calls client.end() when the hook rejects, so a server that refuses the statement does not leave an authenticated backend open once per acquire until max_connections runs out. Both paths are covered by specs that drive a real pg.Pool against an in-process socket server speaking enough of the wire protocol to cut or refuse the pin.
  • assertDataConnectionUtc reads the effective zone back after the data connection initialises and fails boot when the offset is not zero, naming the zone and the remedy. A pin that a pooler or a server-side default overrode would otherwise leave the driver reading every naive timestamp as UTC while the server kept writing another zone, permanently and with nothing to notice. The offset is sampled at two instants six months apart rather than once: Europe/London, Europe/Lisbon and Atlantic/Canary sit at +00 from late October to late March, so a single reading passes an unpinned session for half the year, and from the last Sunday in March every DEFAULT now() then lands an hour ahead of what the driver reads back, with no restart in between. The pair is taken relative to now() so it follows the zone's current rules, and UTC, Etc/UTC and GMT still pass with no name list to maintain.
  • Both entry points carry the pin: createBootDataSource (the runtime data connection's only construction point, which also runs the assertion before any migration writes a row) and the migration CLI data source. The boot advisory-lock client does not: it issues only pg_advisory_lock/unlock and never touches a timestamp.
  • Docs: the column-type section of docs/05, the lease-zone caveat in docs/13 (which this narrows to clock skew, with the mixed-version exception spelled out), the multi-node requirement list in docs/10, the restore runbook in docs/11, and the backup/restore transfer note in docs/14. TZ is explicit in both bundled Compose files and listed in .env.example.

Behaviour and upgrade impact: SQLite is untouched, and a PostgreSQL deployment already running in UTC sees no data move. A PostgreSQL deployment behind a pooler that drops session state now fails to start instead of storing one convention and reading another; the boot error names ALTER DATABASE ... SET TimeZone='UTC' as the alternative. On a gateway that ran outside UTC, the twelve columns the app writes itself now read as shifted by the old offset. Nine are nullable columns with no default; the other three, lid_mappings.updatedAt, chat_states.updatedAt and baileys_stored_messages.createdAt, carry a DEFAULT now() that never fires, because their only writer is an upsert that passes the value, and a spec against a real server pins that classification rather than leaving it to a reading of TypeORM. The [Unreleased] upgrade notes list all twelve, give a conversion that resolves daylight saving per row, separate them from the eighteen genuinely server-written createdAt/updatedAt columns that must not be converted on a UTC server, and split the archive case by source: a SQLite-made archive landed correct UTC rows beside local-time ones, while a PostgreSQL-made archive taken by an off-UTC gateway moved the whole table backward by the offset once per restore, so no row in it is correct and the conversion for the app-written columns would move those rows a further offset the wrong way.

Verified against a real postgres:16-alpine with the process on TZ=Asia/Jakarta and the server's own default zone set to Asia/Jakarta, so both halves of the old split are exercised: the four backup transfer directions (PostgreSQL to PostgreSQL, a repeated restore, a SQLite-made archive onto PostgreSQL, and a PostgreSQL-made archive onto a real SQLite database), a retention sweep through WebhookOutboxService.pruneSettled, a today count through StatsService.getOverview, a live lease carried across a restore, the three app-bound create/update dates, and the boot assertion against a session deliberately set to Europe/London. Each of the three pins was reverted in turn and the suite fails without it. The CI gate set, including the PostgreSQL job's step, was run locally.

Fixes #1624

@rmyndharis
rmyndharis force-pushed the fix/pg-utc-timestamps branch from 2a62538 to dd129d6 Compare September 16, 2026 00:50
A `timestamp without time zone` column carries no zone, and three
writers filled these columns with three different conventions: the
driver bound a JS Date as the gateway's local wall time, parsed a naive
value back the same way, and `DEFAULT now()` behind most
@CreateDateColumn/@UpdateDateColumn wrote the server session's zone.
Off UTC one row therefore held two conventions, which is why a restore
shifted every stamp by the host offset and shifted it again on each
further restore, why a retention `LessThan(cutoff)` deleted rows the
window did not name, and why a lease written by a node in another zone
read as lapsed.

Bind as UTC, parse the scalar timestamp OID as UTC (the array OID keeps
its own parser), and have every pooled connection set its session
TimeZone, which is what makes the server-side default UTC too. That
statement goes through pg-pool's own connect hook rather than a Client
subclass, so it runs with the pool's error listener already attached
(a socket that dies mid-statement fails the acquire instead of ending
the process) and the pool ends the client when the server refuses it
instead of leaking an authenticated backend per acquire.

The session is read back at boot and a connection that is not on UTC
fails rather than storing one convention and reading another. The
offset is sampled at two instants six months apart, because a zone such
as Europe/London reads +00 from late October to late March: one reading
would accept a session that is not pinned and then write every summer
value an hour ahead of what the driver reads back. Both entry points
carry the pin: the runtime factory that builds the data connection and
the migration CLI data source.

Existing data does not move; the upgrade notes derive which columns
shift, give the conversion, and say where it cannot be made safe. Three
columns whose schema default never fires, because their only writer
passes the value, are classified with the app-written ones rather than
with the server-written ones.
@rmyndharis
rmyndharis force-pushed the fix/pg-utc-timestamps branch from dd129d6 to 22aa615 Compare September 16, 2026 00:59
@rmyndharis
rmyndharis merged commit 0c59786 into main Sep 16, 2026
9 checks passed
@rmyndharis
rmyndharis deleted the fix/pg-utc-timestamps branch September 16, 2026 01:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Restoring a backup into PostgreSQL shifts timestamps by the host UTC offset

1 participant