PostgreSQL 19 fixes#180
Open
devrimgunduz wants to merge 2 commits into
Open
Conversation
1. rum: fix build failure against PostgreSQL 19 (Int8GetDatum() removed)
Symptom
-------
Building RUM against PG 19 fails for every translation unit that
includes rum.h (rumbulk.c, rumentrypage.c, rumdatapage.c, rumsort.c,
rumbtree.c, rumtsquery.c, rum_ts_utils.c, rumget.c, ...):
src/rum.h: In function 'rumDataPageLeafRead':
src/rum.h:1022:57: error: implicit declaration of function
'Int8GetDatum'; did you mean 'UInt8GetDatum'?
[-Wimplicit-function-declaration]
1022 | item->addInfo = Int8GetDatum(*ptr);
| ^~~~~~~~~~~~
| UInt8GetDatum
make: *** [<builtin>: src/rumbulk.o] Error 1
(repeated for every other .c file that includes rum.h)
Cause
-----
rum.h's inline helper rumDataPageLeafRead() uses Int8GetDatum() to
convert a raw single-byte pass-by-value attribute into a Datum. In
PostgreSQL 19, Int8GetDatum() is no longer declared by postgres.h, so
the compiler treats it as an undeclared (implicit) function, which is
a hard error under this build's warnings-as-errors flags.
Fix
---
Replace Int8GetDatum(*ptr) with CharGetDatum(*ptr). Both macros/
inline functions have always had the exact same definition
`((Datum) (X))` for a single-byte value -- the only difference is the
formal C parameter type (int8/signed-char vs. char) -- so the
generated Datum is bit-for-bit identical. CharGetDatum() is present
in postgres.h in every supported PostgreSQL release (9.6 through 19),
so this is a safe, non-version-gated, one-line fix, unaffected by
anything below.
No other file in the tree references Int8GetDatum().
--------------------------------------------------------------------
2. btree_rum: fix -Wimplicit-fallthrough=5 warning in
rum_btree_extract_query() without depending on PostgreSQL's own
(repeatedly-changing) fallthrough macro
This went through several rounds as PostgreSQL's internal macro kept
shifting under us. Documenting the whole trail because it explains
why the final fix deliberately avoids PostgreSQL's macro altogether.
Round 1 -- gcc build, PG19
---------------------------
src/btree_rum.c:114:43: warning: this statement may fall through
The BTGreaterEqualStrategyNumber/BTGreaterStrategyNumber case
intentionally falls through into BTEqualStrategyNumber, originally
marked with a `/*FALLTHROUGH*/` comment. This build uses
`-Wimplicit-fallthrough=5`, the strictest level, at which GCC ignores
comment-style annotations and only recognizes the real
`__attribute__((fallthrough))`. First fix: pg_attribute_fallthrough(),
PostgreSQL's wrapper for that attribute (present since PG12). Cleared
the gcc build.
Round 2 -- clang/LLVM bitcode build, PG19
-------------------------------------------
src/btree_rum.c:115:25: error: implicit declaration of function
'pg_attribute_fallthrough'
The PGDG spec also runs a clang `-emit-llvm` pass to produce JIT
bitcode. Under that pass, on PG19, pg_attribute_fallthrough() was
undeclared: PG19 renamed it to pg_fallthrough(). Fix: version-gate on
PG_VERSION_NUM and use the new name on PG19+.
Round 3 -- calling convention, PG19
--------------------------------------
src/btree_rum.c:26:42: error: expected expression before ')' token
26 | #define RUM_FALLTHROUGH() pg_fallthrough()
PG19's pg_fallthrough turned out not to be a renamed drop-in -- it's a
bare macro that expands directly to `__attribute__((fallthrough))`
and must be written without call parens (`pg_fallthrough;`, not
`pg_fallthrough();`). Fix: version-gated macro body (with vs. without
parens) behind a single, unchanged call site.
Round 4 -- clang/LLVM bitcode build, PG18 (this round)
----------------------------------------------------------
src/btree_rum.c:34:27: error: implicit declaration of function
'pg_attribute_fallthrough'
Even on PG18 -- where pg_attribute_fallthrough() has existed since
PG12, and where the gcc build accepted it fine -- the clang
`-emit-llvm` bitcode pass does not reliably declare it. So the
problem isn't just PG19 renaming things; PostgreSQL's own fallthrough
macro is not a reliable target across the gcc/clang x PG12-19 matrix.
Final fix
---------
Stop depending on PostgreSQL's macro entirely. Detect compiler
support for the attribute directly, the same way PostgreSQL's own
headers do internally, and supply our own wrapper:
#if defined(__has_attribute)
#if __has_attribute(fallthrough)
#define RUM_FALLTHROUGH() __attribute__((fallthrough))
#endif
#endif
#ifndef RUM_FALLTHROUGH
#define RUM_FALLTHROUGH() ((void) 0)
#endif
RUM_FALLTHROUGH() expands to exactly what PostgreSQL's own macros were
trying to give us (`__attribute__((fallthrough));` as a null
statement) when the compiler supports it, and degrades to a harmless
no-op on any (now essentially hypothetical) compiler that doesn't.
This is independent of PG_VERSION_NUM, independent of gcc vs. clang,
and independent of the build pass (object file vs. LTO bitcode), so
it can't be broken again by a future PostgreSQL header change.
Testing
-------
- Verified the macro's preprocessor expansion with `gcc -E`: produces
`__attribute__((fallthrough));` exactly.
- Extracted the actual patched macro block and switch statement into
a standalone test file and compiled it with
`gcc -Wall -Wimplicit-fallthrough=5 -Werror -c`: clean build, zero
warnings, zero errors.
- Confirmed this is the only fallthrough site flagged in any of the
warning/error output provided across all four rounds; no other
`/*FALLTHROUGH*/`-style comments exist in btree_rum.c.
- Combined patch dry-run applied with `patch -p1` against the
originally uploaded (pre-any-fix) src/rum.h and src/btree_rum.c and
applies cleanly as a single diff.
Hacked by Claude, tested by me.
…exes with real addInfo data against PostgreSQL 19
Symptom
-------
Regression test rum.sql crashes the backend partway through:
CREATE INDEX failed_rumidx ON test_rum USING rum (a rum_tsvector_addon_ops);
-- (this one is *supposed* to fail -- see below)
...
create index on test_rum_addon using rum (a rum_tsvector_addon_ops, id)
with (attach = 'id', to='a');
-- this one is a properly-configured addon index and should succeed
TRAP: failed Assert("tupleDesc->firstNonCachedOffsetAttr >= 0"),
File: "indextuple.c", Line: 244
resolved via addr2line to:
rumentrypage.c:262 (entryLocateLeafEntry, via inlined rumtuple_get_key)
ruminsert.c:440 (rumEntryInsert)
ruminsert.c:682 (rumbuild's "dump remaining entries" loop)
i.e. during a normal, successful index build, while writing entries
(that legitimately carry addInfo) into the entry B-tree.
Cause
-----
PostgreSQL 19 added a tuple-deformation speedup that gives every
TupleDesc a small offset cache (TupleDescData->firstNonCachedOffsetAttr
and a parallel compact_attrs array). Any TupleDesc that's hand-built
via the classic CreateTemplateTupleDesc() + TupleDescInitEntry()
pattern -- rather than obtained from the relcache/typcache, which
already finalizes it -- must now also be finalized with the new
TupleDescFinalize() before it's used to form or read tuples. The
struct's own doc comment says so directly ("TupleDescFinalize() must
be called on it"), and the assertion RUM is hitting carries the
comment "Did someone forget to call TupleDescFinalize()?" in
PostgreSQL's own source.
RUM's initRumState() (rumutil.c) builds one internal TupleDesc per
indexed column -- state->tupdesc[i] -- using exactly this
CreateTemplateTupleDesc()/TupleDescInitEntry()/
TupleDescInitEntryCollation() pattern, to describe the internal
[(column#), key, addInfo] layout used for RUM's own entry tuples.
On PG19 this TupleDesc is never finalized, so the first time it's
used to form (index_form_tuple) or read back (index_getattr) an
entry tuple that actually carries an addInfo value, PostgreSQL's
core tuple-deformation code hits the new assertion.
This only reproduces with real addInfo data flowing through, which
is why the earlier, simpler rum.sql/rum_tsvector_ops index (no
addInfo) and the deliberately-broken failed_rumidx (which errors out
before ever writing an addInfo-bearing entry) don't trigger it -- only
the properly-configured rum_tsvector_addon_ops index with attach/to
options, later in the same script, does.
Fix
---
Call TupleDescFinalize(state->tupdesc[i]) once each per-column
TupleDesc is fully built (after all TupleDescInitEntry/
TupleDescInitEntryCollation calls for that column, including the
optional addInfo attribute), for both the single-column and
multi-column layouts. Gated on PG_VERSION_NUM >= 190000 since
TupleDescFinalize() doesn't exist before PG19, and isn't needed
there -- pre-19 TupleDesc has no such cache to initialize.
Confirmed CreateTemplateTupleDesc() is only used in this one file
across the whole extension (grepped the full uploaded source tree),
so no other hand-built TupleDesc needs the same fix.
Testing
-------
Traced the exact crash site via the addr2line output provided
(rumentrypage.c:262, ruminsert.c:440, ruminsert.c:682) against
PostgreSQL's own tupdesc.h/heaptuple.c comments, which describe this
exact assertion and its cause verbatim. Confirmed via grep that
rumutil.c's initRumState() is the only place in the extension that
calls CreateTemplateTupleDesc().
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
These two commits fixes build errors against PostgreSQL 19 and also fix a crash that appears during regression tests.
Fix server crash (Assert failure) building rum_tsvector_addon_ops indexes with real addInfo data against PostgreSQL 19
Hacked by Claude, tested by me.
Per: pgdg-packaging/pgdg-rpms#213