Merge dev → main: cortex aingle_path, STM oversized-entry fix, byte-bounded ingest chunking - #134
Merged
Conversation
…otes (#129) * feat(cortex): multiple named MCP credentials The MCP-over-HTTP auth middleware accepted exactly one live bearer token, so a host had to share a single credential across every connected client and a revoke severed all of them at once. The middleware now checks the presented bearer against a live SET of accepted tokens (AppState::set_mcp_tokens / mcp_tokens_snapshot), read per request, so a host can hand each client its own named credential and revoke one without disturbing the rest. set_mcp_token remains as a single-credential convenience wrapper; an empty set fails closed (every bearer rejected). Regression tests cover multi-member acceptance, single-member revocation, and the fail-closed empty set. Bumps aingle_cortex to 0.7.5. * feat(cortex): aingle_path, shortest verified connection between two notes Answers 'how do these two notes relate' with an evidence chain instead of an assertion: BFS over the typed vault graph, where link edges come from every links_to triple (traversed both directions) and semantic edges from the verified neighborhoods of the two endpoints, so unlinked topics still meet through the link fabric between them. Every hop is typed (link|semantic) and carries its similarity score and signed-provenance anchor when available. Endpoints resolve with the same rules wikilinks use (exact path or bare name). Bounded search: hop budget clamped to 6 (default 4), 2000-node expansion cap. The MCP layer hides any chain that crosses a policy-hidden note, reporting no connection rather than leaking the hop. Seven unit tests cover direct links, undirected traversal, hop budgets, name resolution, identity, and honest not-found. * style(cortex,ineru): cargo fmt
…132) Bulk ingest of a large payload produces one MemoryEntry whose serialized size alone exceeds the STM memory budget (1MB in the standard profile). store() tried to prune space for it, but an entry that alone overflows the budget can never be made to fit, so once the STM emptied it returned a hard "STM memory capacity exceeded" error and aborted the ingest. The byte budget is a pruning target, not an admission gate: STM is a transient buffer that consolidates into LTM (which is bounded by count, not bytes). store() now prunes only while pruning makes progress and admits the entry over budget once it cannot free more. This also fixes a latent infinite loop: when every resident entry was already consolidated (prune_one leaves those in place) the prune loop spun forever. prune_one now returns whether it evicted anything so store() can stop. Covered by test_oversized_entry_is_accepted_not_rejected and test_store_terminates_when_only_consolidated_entries_remain.
…rflow memory (#133) Chunking split source purely by line count: a minified or one-line file (JS, CSS, JSON, base64, SVG, generated data — common in real repos) is a single line and became one chunk holding the whole file, multiple megabytes wide. That single MemoryEntry could exceed a memory budget on its own and stall bulk ingestion of large repositories. chunk_fixed now caps every chunk at MAX_CHUNK_BYTES (16 KB): it grows a window up to `window` lines but closes early once the byte budget would be exceeded, and a single line larger than the cap is split at char boundaries into capped pieces, all mapped to that line's number. chunk_markdown routes any section that is long by line count OR by bytes through the byte-aware path, so a short section holding one enormous line (e.g. an embedded base64 image) is bounded too. Normal files chunk exactly as before (50-line windows, a few KB each). Covered by five new chunk.rs unit tests plus the existing ingest suite.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrates
devintomainahead of the next release.Carries:
feat(cortex):aingle_path— shortest verified connection between two notes.fix(ineru): STM tolerates a single entry larger than its byte budget (same change as fix(ineru): STM tolerates a single entry larger than its byte budget #131 already onmain; identicalstm.rscontent, merges without conflict).feat(ingest): byte-bounded chunking so large/minified files never overflow memory — enables ingesting large repositories.mainalready carries the STM fix via #131, so the only net-new code is the cortex path tool and the byte-bounded chunker. Standard merge commit per the dev→main convention.