Skip to content

Complete jq semantics and make evaluation allocation-aware - #6

Merged
jcelerier merged 1 commit into
mainfrom
jq-language-lazy-evaluation
Sep 7, 2026
Merged

jcelerier merged 1 commit into
mainfrom
jq-language-lazy-evaluation

Conversation

@jcelerier

Copy link
Copy Markdown
Member

Problem

The evaluator accepted only part of jq's expression grammar, diverged on several streaming/numeric/error cases, and deep-copied pass-through values at every coroutine boundary. std::function targets and coroutine frames also used the heap without an explicit evaluation lifetime.

Language and semantic changes

  • direct recursive-descent compilation with local action trees
  • if / elif / else, try / catch, select, and, or, and //
  • semicolon-separated filter arguments and the any, all, first, last, isempty, nth, limit, skip, and range arities
  • object-value pipelines, computed/streamed slices, nested interpolation, and jq comments
  • corrected output order, optional-error scope, lazy early termination, array to_entries, and object map
  • exact signed-64-bit comparison/negation, jq NaN equality/ordering, modulo boundaries, and UTF-8 split/slice behavior
  • preserve output emitted before a later error, including lazy object construction

Lazy ownership and allocation

  • coroutine yields distinguish borrowed and owned values; forwarding no longer materializes pass-through arrays/maps/strings
  • prevent accidental promise aggregate initialization from copying coroutine inputs
  • caller-backed evaluation_context routes coroutine frames and allocator-aware values through bounded storage
  • strict contexts throw on exhaustion; explicit upstream resources retain unbounded compatibility
  • take() retains within the selected context; persist() detaches escaping values to heap storage
  • exception payloads persist independently of evaluation storage
  • compiled actions use stable pointer-owned smallfun nodes instead of recursive std::function targets
  • stable index sorting avoids std::stable_sort's hidden allocator

Verification

  • standalone build and CTest: 3/3 targets pass
  • 356 generated conformance cases pinned to official jq 1.8.2
  • ownership tests cover large borrowed inputs, arena reuse, nested evaluation, escaped results/errors, strict exhaustion, and frame deallocation
  • allocation smoke: 0 global heap allocations inside a bounded context for identity, length, selection/projection, map, sort, unique, sort_by, group_by, and long literals
  • actual score boundary smoke passes proper-map selection, callback-retained output lifetime, and callback-triggered program replacement

The allocator-aware OSSIA conversion support used by the score addon is proposed separately in celtera/avendish#202.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant