fix(rt): reserve stack slots in Table raw operations - #39
Merged
Conversation
raw_set/raw_get push up to three stack slots (plus the extra copy value_from_stack adds for reference types) without calling lua_checkstack. When the caller's call-info frame margin is already thin — for example after a long sequence of VM operations, like compiling many scripts in one VM — lua_pushvalue trips the top < ci.top LUAU_ASSERT and aborts the process (EXC_BREAKPOINT) instead of returning an error. Add an ensure_stack helper that reserves the needed slots up front and returns a MemoryError when the reservation fails, and use it in raw_set and raw_get.
Owner
|
@vi2q Thank you for all 3 PRs. I will review them and merge them. |
…t claim
- Match the crate's existing convention (Function::call, Thread::resume,
callback results): a stack reservation failure is a catchable
RuntimeError("stack overflow: ..."), not a MemoryError.
- Rename/re-comment the new test: it exercises sustained raw traffic, but a
C frame always gets LUA_MINSTACK headroom, so it does not by itself drive
the margin low enough to trip the assertion.
Owner
|
Verified and pushed a small follow-up to this branch (2edef5f). The guard itself is right and it's the convention the rest of the crate already follows — Two changes:
Merging it regardless: it's the correct C-API discipline, it's what mlua does via |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Table::raw_setandTable::raw_getpush up to three stack slots without callinglua_checkstackfirst:value_from_stackalso pushes one extra copy for reference types (Table / Function / etc.) insideraw_get. When the caller's current call-info frame margin is already thin,lua_pushvaluetrips the(*L).top < (*(*L).ci).topLUAU_ASSERT and aborts the process (EXC_BREAKPOINT / SIGTRAP) instead of returning aResult::Err.Real-world reproduction: loading a scene that compiles ~35 scripts in one VM, then probing the script environment with
raw_getfrom the host, crashed with the assert. The host cannot defend itself — there is no public API to check the remaining frame margin, and the failing push is inside luaur-rt.mlua is unaffected: its raw accessors run under
ensure_stack/StackGuard.Fix
Add an internal
ensure_stackhelper and reserve the needed slots up front:raw_setreserves 3 slots;raw_getreserves 3 (table + key + the copyvalue_from_stackmay push). A failed reservation surfaces asError::MemoryErrorso embedders can recover.The same pattern likely applies to other raw accessors (
set_metatable,raw_insert,raw_pop, …); I started with the two that crashed in practice and kept the change reviewable. Happy to extend the helper across the remaining stack-touching APIs in this PR or a follow-up.Testing
test_raw_table_ops_survive_a_thin_stack_margininmlua_core.rs: 2000raw_set/raw_getpairs, a 150-deep VM recursion, then another 500 raw operations — all after interpreter work that drains the frame margin.cargo test -p luaur-rt: 248 passed, 0 failed.Found while embedding luaur-rt in a game engine (scene loading compiles every script into one VM; the probe step crashed before this fix).