Release 0.8.0 — context-aware memory delivery, and the evidence that proved it - #10
Merged
Merged
Conversation
Two commands and the pass that closed their limitations, landing in one commit because none of it was ever committed. The 0.7.0 that introduced `devia context` and `devia contribute` never had a source state of its own -- 0.8.0 rewrote the surfaces it added before either reached npm -- so it is not tagged and not published. CHANGELOG keeps its section as the record of what those commands were when they were written, marked never published. npm goes 0.6.0 to 0.8.0. `devia context` selects the smallest sufficient context for one task. The corpus is addressable items, not files: a rule, a memory section split at its heading, one never/always line, one gap or debt row, one impact-map duty. Five tiers decide what the context is, a token budget decides how much fits, and a blocking constraint is admitted before the budget is consulted. `devia contribute` turns a devia problem hit inside somebody's repository into an issue or a pull request under two constraints: a candidate is eligible only because devia re-ran the recorded invocation in a minimal fixture and observed the behaviour (AGT-012), and nothing of the user's repository is uploaded -- secrets, addresses, paths and account names are redacted on the way in and the payload is re-scanned before any upload (PRIV-005). A surviving secret shape blocks rather than warns. The hardening is what the benchmark forced: - A target is not a floor. Target, mandatory floor and selected now travel together, and the promise is explicit per mode: advisory may exceed the target and says so, strict never does. Compression buys mandatory items back toward full text in relevance order, so a larger target always returns more text -- the first implementation did this backwards and that is now a test. `devia check` gains CTX-BUDGET (P2). Closes G11 - The impact map is a router, not only a checklist. A declared change type routes the domains of the memory files it names, so a project that invents `new_consent_record` routes as well as a built-in does. Closes D12 - Evidence is bound to the experiment that produced it. Every run records a digest of the fixture and of the bin/ + src/ that ran it; a fix has to agree about the first and disagree about the second. Editing the fixture until it passes now reports the reproduction changed and stays at `reproduced`. Closes D10 Also fixed: the benchmark rebuilt its corpus for every combination, forty times slower than the thing it measures. Read once per shape, cloned per run: 79s to 1.5s. Version: package 0.8.0, standard 0.2.0 (AGT-012, AGT-013, PRIV-005). Verified: 135 tests pass; validate clean (141 rules, 551 links, index current); check P0 clear (15 pass, 1 warn OPS-001, 3 skip); memory validate 19 pass. Benchmark 352 runs -- critical-rule recall 100%, routing accuracy 100%, strict budget compliance 176/176, advisory over target 68/176 and in all 68 selected equals the mandatory floor to the token. Cost per correct decision is not measured and the output says so.
CI caught it on ubuntu node 22 and nowhere else: `not ok 58 - the benchmark passes: the saving never costs a blocking rule`, failing on `Unterminated string in JSON at position 146032`. Not a selection defect -- the report is 182 kB and the test was reading the first 146 kB of it. `console.log` then `process.exit()` loses whatever is still buffered. On Windows stdout to a pipe is synchronous, so the write always completed; on POSIX it is asynchronous and the exit cut it off once the report outgrew the pipe buffer. ubuntu node 20 won the race, node 22 did not. A test that passes on three of four runners because of timing is the worst kind of green. `fs.writeSync(1, ...)` blocks until the bytes are gone, so the exit on the next line cannot truncate it. The failure path drops to `process.exitCode`, which is what bin/devia.mjs and src/lib/ui.mjs already use -- the CLI never calls process.exit() and was never exposed to this. Scope: scripts/benchmark-context.mjs only. Nothing in bin/ or src/ changes, so the published 0.8.0 CLI is unaffected and the version is not bumped. The 0.8.0 tarball on npm carries the unfixed script; no adopter runs it. Verified: `node scripts/benchmark-context.mjs --json` emits 182585 bytes that parse whole -- ok true, 352 rows, 0 failures. 135 tests pass. Benchmark unchanged at 352 runs, recall 100%, routing 100%, strict 176/176.
The previous commit was the right diagnosis and the wrong remedy. CI failed again on ubuntu node 22, at byte 146032 of 182402 -- the same byte as before. An identical cut point is not a race; a single fs.writeSync on a non-blocking pipe writes what currently fits and returns short, and the return value was being discarded. The remaining 36 kB were never written. Loop until every byte is gone, treating EAGAIN as "the reader has not drained yet" rather than an error. Windows never showed this because stdout there is synchronous and the first call always completed; a redirect to a file did not show it either, because a file fd is not non-blocking. Only a pipe -- which is exactly what execFileSync gives the test -- could. Still scripts/benchmark-context.mjs only. bin/ and src/ are untouched and the published 0.8.0 CLI remains unaffected. Verified: the report through a pipe is 182402 bytes and parses whole -- ok true, 352 rows, 0 failures. 135 tests pass locally, but the defect only reproduces on POSIX, so ubuntu is the run that decides this.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Releases 0.8.0. Three commits on top of
ddbdcf5(main, v0.6.0):a7175d8is the release,07bbf5aandaf1e029fix a defect CI found on this branch after publication. CI is green on all four runners.0.7.0 — skipped, not published, documented
0.7.0 prepared
devia contextanddevia contributebut was never committed: no tag, no branch,no stash, nothing in the reflog held a source state for it, and 0.8.0 rewrote the surfaces it
introduced (
context.maxTokens→context.budget/context.mode, strict compression, theCTX-BUDGETgate, impact-map routing, evidence hashing, and a benchmark rewritten from 6scenarios to 352 runs) before either reached npm.
Publishing it would have meant hand-carving a source tree that never existed and was never
tested, then making its changelog claims true by deduction — for a permanent npm artifact. It is
skipped instead. Its
CHANGELOGsection is preserved and marked(never published)with thereason and a pointer to 0.8.0, so an adopter looking for
0.7.0on npm understands. Nov0.7.0tag was created. npm goes 0.6.0 → 0.8.0.0.8.0 — published
@schneiderjoseph/devia@0.8.0,dist-tag latest→0.8.0v0.8.0(annotated), ona7175d8— the exact commit whose contents were publishedThe published tarball's shasum,
cbaf12cc426128862c68938d654074909b0d79a0(449 files,590,948 B unpacked), is identical to the tarball packed from this branch and exercised before
publication — the pre-publication verification applies to the published artifact bit for bit.
Tests performed
npm testnpm run validatenode bin/devia.mjs check --root .OPS-001, lint not referenced) · 3 skip · P0 clear · exit 0node bin/devia.mjs validateNo test or threshold was modified to reach these numbers.
Benchmark verification
npm run benchmark:context— 352 runs (4 corpus shapes × 11 task types × 4 targets × 2 modes):In all 68 advisory overruns
selectedequals the mandatory floor to the token. Token counts areestimates from
src/lib/tokens.mjs, not a tokenizer's output. Cost per correct decision is notmeasured.
Package verification
Tarball: 449 files, 191.9 kB packed. Audited for
.git,.env,node_modules,tests/,.devia/, contribution payloads,reader.html,*.pem,*.key, logs and fixtures — nonepresent. Top level is
bin,src,rules,checklists,standard,compliance,schema,templates,skills,scriptsand the governance docs, matchingfilesinpackage.json.Installed twice into clean temporary directories — once from the local tarball before publishing,
once from the registry afterwards — and exercised both times:
Release notes
CHANGELOG.mdcarries the 0.8.0 entry and the annotated 0.7.0 entry. The GitHub Release notesstate explicitly that the contribution workflow is not automatic (
submitwrites the payloadand prints the
ghcommand), that token counts are estimates, and that cost per correct decisionis not measured.
07bbf5a+af1e029— a defect CI found after publicationThe first CI run on this branch failed on one of four runners — ubuntu-latest, node 22 —
with
not ok 58 - the benchmark passes: the saving never costs a blocking ruleandUnterminated string in JSON at position 146032. Windows node 20, Windows node 22 and ubuntunode 20 all passed, and so did every local run.
It is not a selection defect.
scripts/benchmark-context.mjs --jsonemits a ~180 kB report andthe test reads it through a pipe. Two things were wrong, and the first fix only found half of it:
console.logfollowed byprocess.exit()discards whatever is still buffered. On Windowsstdout is synchronous so the write always completed; on POSIX it is asynchronous and the exit
cut it off.
07bbf5areplaced it with a synchronous write.identical cut point is not a race. A single
fs.writeSyncon a non-blocking pipe writes whatcurrently fits and returns short, and the return value was being discarded; the remaining 36 kB
were never written.
af1e029loops until every byte is gone, treatingEAGAINas "the readerhas not drained yet" rather than an error.
A redirect to a file never showed this either, because a file descriptor is not non-blocking.
Only a pipe — exactly what
execFileSynchands the test — could.Scope and consequence. Both commits touch
scripts/benchmark-context.mjsand nothing else.bin/andsrc/contain noprocess.exit()call at all — the CLI usesprocess.exitCodethroughout and was never exposed to this. The published 0.8.0 CLI is unaffected and the version
is not bumped.
One divergence worth stating plainly: the 0.8.0 tarball on npm carries the unfixed benchmark
script. It is a development script no adopter runs, the table form of
npm run benchmark:contextwas never affected, and the measurements are identical before and after. Ifyou would rather npm and
mainmatch exactly, that wants an 0.8.1 — it is not a correctnessreason to republish.
Not verified
devia contribute submit --yes, which is the only network path in devia — deliberately notexercised
initwrites; no claim of universal agentcompatibility is made anywhere in the release notes