Skip to content

fix: seven CLI defects a real end-to-end run exposed - #46

Draft
david-hudec-networg wants to merge 7 commits into
TALXIS:masterfrom
david-hudec-networg:fix/add-scaffolds-invalid-documents
Draft

fix: seven CLI defects a real end-to-end run exposed#46
david-hudec-networg wants to merge 7 commits into
TALXIS:masterfrom
david-hudec-networg:fix/add-scaffolds-invalid-documents

Conversation

@david-hudec-networg

@david-hudec-networg david-hudec-networg commented Sep 7, 2026

Copy link
Copy Markdown

This is the fixes PR. Seven defects the CLI shipped, all of them hit by
running a five-source discovery engagement through it end to end rather than
found by reading. Its companion #47 is
the improvements PR and proposes language changes; nothing here does.

That split is deliberate. These are bugs, and they should not wait on a design
debate about the other one.

Read it in seven commits

commit what it is
1 three add scaffolds emit shapes their own schema rejects independent of everything else
2 generated templates carried no property information at all the root cause of the add family
3 sequence sections were scaffolded as ID-keyed maps the writer's own defect, visible once 2 lands
4 test: every add scaffold must validate goes green here and nowhere earlier
5 nextid could not be trusted as a source of truth three compounding defects
6 derivedFrom on value streams and capabilities the one document type with no machine-checked provenance
7 ubml walk bookkeeping for a stakeholder review

Each builds on the last. Nothing is written and then rewritten.

ubml add produced a document ubml validate rejects

ubml init proj --minimal && cd proj
ubml add glossary terms
ubml validate .
  :14:11  error  Type mismatch: expected object, got object (/terms/TM01000)
  :18:3   error  Type mismatch: expected array, got object (/categories)

Swept across every type add can create: 13 of 13 failed.

One line in generate-template-data.ts:

properties: [],          // <- every section, every type

TemplateSection never carried properties, so TEMPLATE_DATA held none for
anything and the scaffolder wrote a bare key. TM01000: with an empty body
parses as null. The types that appeared to work only did so because items.ts
happened to carry a hand-written example, and two of those were wrong in other
ways.

Extraction now resolves the item schema from the document schema —
patternProperties for an ID-keyed map, items for a sequence, inline or behind
a $ref. Five shape bugs fell out of the sweep:

was now
add hypotheses root/children as {id, text, …} and arrays HY#####-keyed maps
add process wrote id: PR01000 inside the item dropped; the ID is the key
sequence sections scaffolded as ID-keyed maps empty list, shape in comments
reference placeholders always AC##### prefix from the referenced definition's own pattern
roiAnalyses RO##### ROI##### — the prefix regex was [A-Z]{2}

ubml nextid could not be trusted

Three defects compounding, none of which announced itself.

It consumed the ID it reported. Asking what the next ID would be wrote that
answer back, so three calls gave three answers, each higher.

syncids could not repair a stale cache. It kept only the larger of cached
and supplied, so a scan finding a lower truth was discarded. It printed
AC: 1020 → 11 and wrote nothing — the one case the command exists for.

minStart was applied to a continuation. A workspace numbered from AC00001
was told its next actor was AC01000, leaving a 989-wide hole.

The visible symptom was Highest: AC00008 and Next ID: AC01020 printed one
line apart, both meant.

Two additions

derivedFrom on valueStreams and capabilities — the one element family
whose provenance the validator could not see.

ubml walk — which insight is next, in the order the material was produced,
with the source text beside the claim; and recording the answer. It stops at
bookkeeping: reading a reply, judging an ambiguous answer, noticing a correction
changed the meaning are not a CLI's job. set rewrites one line rather than
reserialising, because a workspace is a document a human is reading.

Tests

291 passed, 3 skipped. typecheck and lint clean.

cli-add-validates.test.ts fails 13 of 13 against the previous behaviour — I
rebuilt from pre-fix sources to confirm the test can fail. id-scanner.test.ts
covers all three nextid defects. cli-walk.test.ts covers ordering, evidence
before claim, ID last, one line changed, and a byte-identical round trip.

Note on ordering

#47 is stacked on this one and shows
these commits too until this merges. Merge this first.

david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
It said nextid scans the workspace. It prefers a cache that goes stale, which is
the opposite of the guarantee that sentence was offering.

The hypotheses scaffold is not the only broken one either - the sweep behind
TALXIS/ubml#46 found all thirteen types emit something their own schema rejects,
so the advice is now validate after scaffolding rather than avoid one template.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Each is independent of the others and of anything else; together they are what
can be fixed without touching how templates are generated.

The hypotheses tree used an {id, text, ...} node and array children where the
schema wants HY-keyed maps. The process wrote an id property inside the item,
where the ID is the key and Process declares none. Both scaffolds were teaching
a shape the bundled validator refuses, which is worse than being merely broken:
the natural reading is that the schema is stricter than the docs, so you hand-
write around a validator that was right.

The entities schema set templateDefaults.entities.type to "business-object".
Entity has no type property and sets additionalProperties: false, so that one
default was enough on its own to keep every scaffolded entities document
invalid.

Verified: hypotheses and process now scaffold and validate clean. entities
still fails on an unrelated defect - the generated templates carry no property
information at all - which is the next change in this series.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@david-hudec-networg
david-hudec-networg force-pushed the fix/add-scaffolds-invalid-documents branch 2 times, most recently from bf47bb1 to 06cc5af Compare September 7, 2026 18:08
david-hudec-networg and others added 3 commits September 7, 2026 20:15
This is the defect behind most of the invalid scaffolds. One line in
transformTemplateData:

    properties: [],          // every section, every type

TemplateSection never carried properties and the transform hardcoded an empty
list, so TEMPLATE_DATA held nothing for anything. The scaffolder's default path
builds an item from section.properties.filter(required), which is always empty,
and writes a bare key:

    terms:
      TM01000:

TM01000: with an empty body parses as null, not an object, and validate rejects
a file add has just written. The types that looked healthy were the ones with a
hand-written example in items.ts; nothing else stood between the generator and
the output.

Extraction now finds the schema for one item of a section - patternProperties
for an ID-keyed map, items for a sequence, either inline or behind a $ref into
types/ - and records name, type, description, required, enum values and default.
resolveRef returns null rather than throwing on a shape it does not recognise,
so an unfamiliar reference degrades to the previous behaviour instead of
breaking generation.

Three things fell out of reading the schema properly. Reference placeholders
were always AC#####, whatever the field accepts, so a scaffolded step link
failed its own pattern check; the prefix now comes from the referenced
definition's own pattern, StepRef declaring ^ST\d{5,}$. Required array and
object properties were filled with the string 'TODO'. And section ID prefixes
were matched with [A-Z]{2}, so roiAnalyses (ROI#####) became RO#####.

Sequence sections are still written as maps after this; that is the next commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
links and glossary categories are YAML sequences in the schema, and the
scaffolder had one way of emitting a section, so it wrote them as maps keyed by
an ID that does not belong there.

They now scaffold as an empty list with the item shape in comments, rather than
as a sample row. A links item requires from and to, and any value there points
at step IDs the user has not created yet, so a populated sample fails
cross-reference validation the moment it is written. An empty list validates and
the comment still teaches the shape - including the correct reference prefix,
which the previous commit made available.

With this, every document type add can create scaffolds a document validate
accepts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Asserts the property that was missing the whole time: scaffold a document of
each type, then validate the workspace it landed in.

It can fail. Rebuilt from pre-fix sources, all 13 cases fail; with the series
applied, all 13 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@david-hudec-networg
david-hudec-networg force-pushed the fix/add-scaffolds-invalid-documents branch from 06cc5af to 3058bd8 Compare September 7, 2026 18:16
Three defects compounding, all of which made the answer wrong without saying so.

It wrote its own answer back to the cache, so asking what the next id would be
consumed it. Three calls in a row gave three answers, each higher. A query that
mutates the state it reports has no honest use.

writeIdStats kept only the larger of the cached and the supplied value, so a
scan could never lower a stale entry - which is precisely what syncids exists to
do. Reported "AC: 1020 -> 11" and wrote nothing.

And minStart was applied even when ids already existed, so a workspace numbered
from AC00001 was told its next actor was AC01000. That floor is documented as
the starting number for add templates; applying it to a continuation query
leaves a 989-wide hole and two disjoint ranges in one document.

The consequence was a command that printed Highest: AC00008 and Next ID: AC01020
one line apart and meant both. Following it numbers a model from another
branch's high-water mark.

Tests cover all three: the query is idempotent, a scan can lower a cached value,
and a continuation continues.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@david-hudec-networg david-hudec-networg changed the title fix: ubml add scaffolds documents that ubml validate rejects fix: make the CLI trustworthy for a source-to-model pipeline Sep 7, 2026
Every other element type could cite the insights behind it. Strategy could not,
so a strategy document's provenance was prose the validator never saw - the one
document in a workspace where the pipeline's guarantee did not hold, and no way
to tell from inside it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
david-hudec-networg added a commit to david-hudec-networg/skills that referenced this pull request Sep 7, 2026
The repo's principle is that every workaround a skill has to describe is a
defect in the tools. Four were: add scaffolded invalid documents, nextid could
not be trusted, strategy elements could not cite their evidence, and supersedes
takes a single reference.

Three are fixed in TALXIS/ubml#46 and the prose they justified is gone from the
skills. The fourth stays proposed - accepting a list there is a language
semantics change, not a bug fix.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Which insight is next, in the order the material was produced, with the source
text beside the claim and a position counter. And recording the answer once a
human has given one.

It stops at bookkeeping on purpose. Reading a reviewer's reply, judging whether
an ambiguous answer counts, noticing that a correction changed the meaning
rather than the wording - none of that is a CLI's job. The payload and the state
are mechanical; the judgement is not.

set rewrites one line rather than reserialising. A workspace is a document a
human is reading: its comments, its quoting and its line breaks are content, and
round-tripping them through the serializer would rewrite a file underneath the
reviewer. Tests cover it - one line changes, a comment survives, and setting a
status and setting it back returns the original bytes.

A restatement means a claim repeated from a source already walked, so it points
at an insight belonging to an earlier source. Counting every related link
instead would tell a reviewer "15 insights, 13 restating" on a workspace where
almost nothing restated - worse than silence, because the map exists so they can
pace themselves.

The ordering, the labelled blocks, evidence before claim, position ahead of
identity and the ID last were all prose in the consulting plugin's
validate-insights skill. They are assertions here now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@david-hudec-networg
david-hudec-networg force-pushed the fix/add-scaffolds-invalid-documents branch from 0124970 to faebe65 Compare September 8, 2026 06:57
@david-hudec-networg david-hudec-networg changed the title fix: make the CLI trustworthy for a source-to-model pipeline fix: seven CLI defects a real end-to-end run exposed Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant