pbc: heap corruption crashes large-tree analysis (non-deterministic; original stack-overflow diagnosis was wrong)

[jimmykirk]

Summary

pbc segfaults partway through parsing a large source tree when the process
has the common default 8 MB stack. It is a C-stack overflow, not an OOM, and
raising RLIMIT_STACK is enough to get past it.

I lost a fair amount of time treating this as a memory-size problem, so the
elimination evidence is below in case it saves someone else the trip.

Reproduction

Parsing a tree of ~5,500 .sr* files (a mix of .srd, .srw, .sru) inside
a container with the default 8 MB stack:

pbc --input <tree> --db out.duckdb --sql-worker-python python
-> Segmentation fault (core dumped), exit 139

It dies during phase A0, after a different number of files each run:

816 / 1421 / 2476 / 2577 files parsed before the crash

It is not memory, and not parallelism

Same input, varying only the RTS flags — every one segfaults:

flags	result
-M4G -N4	SIGSEGV, 41s
-M8G -N4	SIGSEGV, 41s
-M12G -N4	SIGSEGV, 45s
-M16G -N1	SIGSEGV, 46s
-M16G -N2	SIGSEGV, 13s
-M16G -N4	SIGSEGV, 44s

A genuine heap overflow aborts cleanly with Heap exhausted, and -M4G would
certainly hit that ceiling first if this were one. The crash ignores the heap
cap entirely, and -N1 rules out a data race in the parallel path.

It is the stack

Changing nothing but RLIMIT_STACK:

stack	result
8 MB (default)	SIGSEGV after 816–2577 of 5473 files
unlimited	5446 of 5473 files parsed

This also explains the non-determinism: how deep the parser recurses depends
on which object it is on when cumulative depth crosses the limit, so the
crash lands on a different file each run, and larger trees fail more often
simply because they offer more chances to hit a deep object.

Workaround

docker run --ulimit stack=536870912:536870912 ...

A finite value rather than unlimited matters: glibc sizes a new thread's
stack from RLIMIT_STACK, but falls back to a fixed 8 MB default when it is
unlimited, so unlimited fixes only the main thread and leaves -N workers
at the broken size.

What I could not narrow down

I could not build a minimal synthetic reproducer. These all parse cleanly at
8 MB, so the deep recursion does not appear to be in expression parsing:

• a 50,000-term flat concatenation ("a" + "b" + ...)
• 30,000-deep explicit paren nesting
• 4,000 trivially-simple generated window objects

Splitting a real tree by file type, at 8 MB stack, also did not isolate it:
the .srd files alone parsed clean, and the .sru files alone parsed clean.
The trigger seems to need a mix. I am not able to share the tree that
reproduces it.

Offer: pbc ships stripped, so the cores I have are addresses only. I am
happy to rebuild with --disable-executable-stripping --enable-debug-info
and post a backtrace — that would presumably identify the recursive function
outright and save you reproducing this at all. Say the word and I will run it.
I can also test any candidate patch against the tree that reproduces it, which
gets you a verification path without needing the source.

Environment

• pbc built from haskell:9.10.2-bullseye, GHC 9.10.2, per the versions in
  the project's own CI workflow
• Linux x86_64, 64 GB RAM, 16 cores
• Container memory limit well above peak usage in every run above

[jimmykirk]

For completeness: with the stack raised, parsing completes but the job still does not finish — it is then OOM-killed in the post-parse phases. That second wall is filed separately as #4, since it looks like an unrelated defect with a different fix.

[jimmykirk]

Correcting this report. The diagnosis above is wrong and the workaround I
gave does not do what I claimed. Details below, along with what it actually is
and one genuinely useful result: GHC 9.12.2 fixes the parsing crash.

1. The workaround I posted is broken

docker run --ulimit stack=536870912:536870912 ...

That pins the hard limit as well as the soft one, so nothing inside the
container can raise it afterwards — ulimit -s unlimited then silently
reports back the cap rather than failing:

$ ulimit -s unlimited
$ ulimit -s
2097152          # the cap, not unlimited

My original experiment ran in a container with the default (unlimited) hard
limit, so it was not testing what I thought. Anyone following that advice gets
a cap, not a fix. Sorry.

2. It is not a stack overflow

It is heap corruption. A build with symbols
(--disable-executable-stripping --enable-debug-info) under gdb, GHC 9.10.2:

Thread 72 "ghc_worker" received signal SIGSEGV, Segmentation fault.
0x00000000004109ed in evacuate ()

#0  evacuate ()
#1  scavenge_block ()
#2  scavenge_loop ()
#3  scavenge_until_all_done ()
#4  gcWorkerThread ()
#5  yieldCapability ()
#6  schedule ()
#7  scheduleWorker ()
#8  workerStart ()

Running the same input with the serial collector (-qg) does not segfault —
it gets all the way through parsing and then the RTS catches the corruption
itself:

pbc: internal error: scavenge_mark_stack: unimplemented/strange closure type 4383578 @ 0x447bf73030
    (GHC version 9.10.2 for x86_64_unknown_linux)

Closure type 4383578 is not a closure type. Same corruption, two symptoms
depending on which collector trips over it.

This also explains the properties that made it look like a stack problem:
non-deterministic (depends on GC timing), indifferent to -M and to
RLIMIT_STACK, worse with larger inputs, and not reproducible from any small
subset. The "8 MB crashes / unlimited survives" A/B in my original report was
nondeterminism across too few runs.

Not a packaging artefact, checked because ABI mismatch is the usual cause:
libduckdb 1.5.4 and GHC 9.10.2, matching this project's own CI, and
satisfying duckdb-ffi >= 1.5.0.0 && < 1.6.

3. GHC 9.12.2 fixes the parsing crash

Rebuilt against haskell:9.12.2. The only obstacle was the cabal bound
base ^>=4.20.1.0, which caps at 9.10's base — relaxing it to < 4.22 was
enough, nothing else objected.

On an identical 18,264-file tree, parallel GC, two runs: all files parsed in
~125s, no segfault. On 9.10.2 the same configuration failed every time at
roughly 2,000 files.

9.12.x carries RTS fixes in this area (WHITEHOLE handling in scavenge
operations, #26205/#26204) and new weakly-pinned-array primops aimed at FFI
safety, either of which could plausibly be responsible. I have not tried to
attribute it to a specific commit.

4. What is still broken

A separate failure remains in "Building taint closure", on both GHC
versions, and that phase consumed 37 GB without completing on a tree this
size.

One caution learned the hard way there. Across both GHC versions:

heap cap	outcome
small (-M14G, -M16G)	clean Heap exhausted
large (-M32G)	SIGSEGV at ~37 GB

I briefly read the clean Heap exhausted as "the crash is fixed". It was not
— the cap had simply stopped the run before the crash could trigger. Worth
flagging in case anyone else benchmarks this: a clean heap exhaustion is not
evidence of a fix.

Summary of the corrected state

• The parsing crash is heap corruption, fixed by GHC 9.12.2.
• Stack limits are irrelevant; disregard the --ulimit advice above.
• A distinct, unresolved problem lives in the taint closure, which looks
  memory-shaped rather than corruption-shaped, and which I am looking at next.

Happy to retitle or close this in favour of a cleaner issue if you prefer — it
has accumulated a wrong diagnosis in the original post, which is not ideal for
anyone finding it later.

[jimmykirk]

Retracting the GHC 9.12.2 claim from my previous comment. It is not
supported, and I should not have posted it.

I based it on two clean full-corpus runs. A later run on the same 9.12.2 build,
on a 6,000-file subset of the same tree, segfaulted during parsing at 2,426
files — the same signature I attributed to 9.10.2. Two lucky runs against a
non-deterministic bug is not evidence of a fix, and I presented it as if it
were. Please disregard that section; I have also removed the claim from the
issue title.

What still stands from that comment, all independently reproduced:

• The --ulimit stack=N:N workaround in the original report is broken — it
  pins the hard limit, so ulimit -s unlimited inside the container silently
  reports the cap.
• The failure is heap corruption, not stack overflow: a symbol build
  faults in evacuate() inside a GC worker thread, and under the serial
  collector the RTS aborts with
  scavenge_mark_stack: unimplemented/strange closure type 4383578.
• Stack size is irrelevant — 8 MB through unlimited all crash.

What I can say about 9.12.2 honestly: it did not obviously make things worse,
and two full-corpus parses completed on it that never completed on 9.10.2. That
is an observation, not a fix. I have not run enough trials to give a failure
rate, and the runs are expensive enough (a surviving run takes tens of minutes)
that I would rather report nothing than guess again.

One further observation that may be more useful than the version question: the
crash lands in a different phase each time — parsing, Materialize: taint reaches + confirmed, and Schema closure have all been observed — which is
consistent with a single corruption being detected wherever the run happens to
be when GC next walks it, rather than with a bug local to any one pass.

Separately, and not a crash: Schema closure appears to be a scalability
problem on its own. On a 6,000-file subset it ran over 40 minutes without
completing or emitting further progress events. I will open that separately if
it turns out to be distinct from the corruption.

[jimmykirk]

Further evidence, plus one confound I should disclose up front.

The strongest argument is not the crash site, it is the language

Two independent RTS-level confirmations that this is heap corruption:

• evacuate() faulting inside a parallel GC worker (gdb, symbol build)
• scavenge_mark_stack: unimplemented/strange closure type 4383578 under the
  serial collector

In a memory-safe language, the heap does not become corrupt on its own. The
candidates are an RTS bug or unsafe code, and the only unsafe surface this
program has is the DuckDB C API via duckdb-simple / duckdb-ffi. That is
where I would look first, ahead of GHC.

Crash sites, with the caveat that they are weak evidence

Observed across runs:

configuration	crash site
default, GHC 9.10.2	evacuate during A0 parsing (~1,700–2,500 files)
serial GC (-qg)	RTS abort in Building taint closure
GHC 9.12.2	SIGSEGV in Schema closure; also in A0 at 2,426 files
debug RTS (-DS)	SIGSEGV at Flushing Phase A appender pool
analysis phases skipped	SIGSEGV at Starting SQL bridge pool (x2, 300s and 552s)

The two most recent configurations both land at a DuckDB boundary — the
appender flush and the SQL bridge pool. That is suggestive, and it is the
reason I am pointing at the FFI.

But I want to be honest that crash location is weak evidence of origin
here: corruption surfaces wherever the collector next walks it, so these
sites may simply be where allocation is heaviest rather than where the damage
is done. The list above is offered as a pattern to check, not a conclusion.

Notable: work is not proportional to the question

Separately, while trying to skip the taint pass I found that emptying the
source list is not sufficient — buildTaintClosure builds the entire relation
through taintPathRelation from every intra-edge, return row and interproc
edge before any source is consulted. With zero sources it still consumed
29.7 GB on an 18,264-file corpus. Gating the whole call was required.

That may be worth a look independently of this bug: a taint query with no
sources should be cheap.

Confound I cannot rule out

GHC itself segfaulted twice while compiling this project on the same machine
(exit 139, at ~55 GB free memory, no MCE/EDAC entries in the kernel log);
cabal build -j1 then succeeded immediately and has been reliable since. I
cannot fully exclude an environmental factor on this host.

Against that reading: pbc's crashes are input-dependent and reproducible —
the same tree crashes across many runs while smaller subsets of it complete
cleanly and repeatedly, and the RTS's own corruption check fires with a
specific invalid closure type. Random hardware faults would not be that
selective. But you should weigh the report knowing the compiler on this box
has also crashed.

Reproduction

An ~4,000-file tree reproduces in roughly 220s. I cannot share the tree, but I
am happy to run any instrumented build, patch, or RTS configuration against it
and report back — that has been the fastest loop so far.

[jimmykirk]

Audited the unsafe surface looking for the corruption. Negative result — everything I could check by inspection appears sound. Posting it so nobody repeats the search, plus one latent hazard worth knowing about.

FFI marshalling — looks correct

• PB.Pipeline.DuckDb.appendTextRows marshals through BS.useAsCString (TE.encodeUtf8 t), which copies into a malloc'd, NUL-terminated buffer kept alive for the call. Not a pointer into the Haskell heap.
• PB.Pipeline.DuckDb.Appender uses alloca/peek/poke conventionally, and brackets end_row so a row can never be left unfinalised.
• duckdb-ffi 1.5.0.0 declares 573 foreign imports, all safe (481 explicit, 92 defaulted) — none unsafe. That rules out the classic "unsafe call, GC moves the argument" failure on both layers.

unsafeCoerce — sound, because the indices are phantom

PB.Compile.IR and PB.Compile.FromSSA coerce through a type-erased table, which was my main suspicion since a bad coercion produces exactly strange closure type. It checks out:

• No constructor of Eff a b carries a value of type a or b — the indices are purely phantom, so coercing Eff () () to Eff () (Either () ()) (which FromSSA does deliberately, and documents) does not change the runtime representation.
• foldFreyd's memo cache is the riskier one, since it coerces the folded result k x y in the target category rather than Eff. But every category it is instantiated at in production is a phantom-indexed newtype:
  • SchFootprint a b = FunctorCtx -> Set SchMorphism
  • NGB a b = Text -> Text -> NamedBuilder Text
  • WB a b = Text -> Text -> WiringB Text
  • TaintEdges a b — record, no a/b payload

Latent hazard, not a live bug

PB.Compile.Interp is the one category whose indices are representationally significant:

newtype Interp a b = Interp { runInterp :: a -> StateT InterpState IO b }

foldFreyd's memo cache coerces cached results between whatever type indices the use sites have. That is safe for the phantom categories above, but for Interp it would be coercing a real function between different argument/result types.

Today PB.Compile.Interp is imported only by compiler/test/EffTermTest.hs, so nothing in production hits it. But the safety of that cache is a property of which categories it is used at, not of the cache itself, and nothing records that. If Interp (or any future index-carrying category) is ever folded through foldFreyd on a term containing shared ELetRef blocks, the coercion becomes unsound — and would present exactly as this issue does.

Might be worth a note at the unsafeCoerce site, or a Typeable check in a debug build.

Where that leaves it

I have eliminated what inspection can reach. The remaining approaches I can see are valgrind against the ~4,000-file reproduction (slow but definitive for C-side errors) or a -debug RTS with more aggressive sanity checking than -DS gave me. Happy to run either if you think one is worth the cycles — or to test a hypothesis of yours, which is likely a better use of the reproduction than more guessing on my side.

[jimmykirk]

Valgrind reproduced it and caught the faulting access. This narrows things considerably, and I think rules out the FFI theory I was pushing.

The run

memcheck against the same ~4,000-file tree, -N1, coslice skipped so it reaches the crash quickly. It crashed in 255s (vs ~220s uninstrumented):

Thread 63 ghc_worker:
Invalid read of size 2
   at 0x4109CC: evacuate (in pbc)
   by 0x40DDFB: scavenge_block
   by 0x1AC3F4A: scavenge_loop
   by 0x1AB7E3D: scavenge_until_all_done
   by 0x1AB8475: gcWorkerThread
   by 0x1A9A7A5: yieldCapability
   by 0x1AA7428: schedule
   by 0x1AA83CC: scheduleWorker
   by 0x1AAC67E: workerStart
Address 0x421cf036ee is in a --- anonymous segment

Process terminating with default action of signal 11 (SIGSEGV)
Bad permissions for mapped region at address 0x421CF036EE

Three things that together say a lot

1. Exactly one error in the whole run. ERROR SUMMARY: 1 errors from 1 contexts — the invalid read above, and nothing else.

2. Zero invalid writes, anywhere. memcheck flagged no out-of-bounds store at any point before the crash. If C code (DuckDB, or the appender marshalling) had scribbled past a buffer, that is exactly what memcheck exists to catch, and it saw nothing.

3. The faulting address is misaligned. 0x421cf036ee mod 8 == 6. GHC heap pointers are 8-byte aligned, so this is not a heap pointer that went stale — it is a value that was never a pointer, being followed as one.

Taken together: the collector walked a closure whose header does not describe what is actually there. The bad value reached a valid heap slot — memcheck cannot see a wrong value written to a legitimate address, which is consistent with it reporting no write error at all.

So this looks like a wrong value in a good slot, not a stray write into a bad one. That points away from the C boundary and toward whatever produced the object.

What this retires

My earlier FFI hypothesis. Combined with the audit further up — useAsCString marshalling copies into malloc'd buffers, all 573 duckdb-ffi imports are safe, no unsafe anywhere — I no longer think the DuckDB boundary is implicated, and I would not spend your time there.

Also worth noting

It reproduced with -N1, so it does not require the parallel collector, even though the trace runs through gcWorkerThread. Earlier runs at -N2 and -N4 crashed the same way. Whatever writes the bad object does not need concurrency to do it.

Reproduction is now cheap

255s under valgrind, ~220s without. If you have a hypothesis you want tested — an assertion, a -debug RTS flag, a patch, a narrower phase — I can turn it around quickly against the tree that reproduces it. That is probably worth more than further guessing from me.