Skip to content

fix: enrich docs for every operation, not the first three paths - #3

Open
Bradenream wants to merge 2 commits into
masterfrom
braden/fix-rewrite-docs-slice/COR-0
Open

fix: enrich docs for every operation, not the first three paths#3
Bradenream wants to merge 2 commits into
masterfrom
braden/fix-rewrite-docs-slice/COR-0

Conversation

@Bradenream

@Bradenream Bradenream commented Aug 11, 2026

Copy link
Copy Markdown

Problem

scripts/rewrite_docs.ts is the second half of yarn codegen. It iterated Object.values(document.paths).slice(0, 3), so x-vf enrichment reached 7 of 151 operations. The slice was present in the file's initial commit (163f316) with no other limiting logic — debug residue, not a deliberate bound.

Correction to how this was reported: enrichment is not "reaching the first three paths and skipped elsewhere." No operation defines cli.example or cli.description — zero of 151. The script today is a complete no-op; running it changes zero bytes. The populated ### Examples blocks in docs/ are Speakeasy's output, not this script's. This PR is plumbing for a spec shape that does not exist yet.

Why deleting the slice is not enough

It crashes yarn codegen. Three problems were masked:

  1. replace(' ', '_') substitutes only the first space. 69 of 151 commands are three-word, producing docs/vf_api-tool_variable create.md. cmd/gendocs/main.go:98 builds the real filename with strings.ReplaceAll(cmd.CommandPath(), " ", "_"); this now matches.
  2. x-speakeasy-ignore: true operations have no docs page. All 12 are create-many batch endpoints. Iterating them raises ENOENT and aborts.
  3. OperationMetadata.parse throws on an operation with no CLI metadata. Not reproducible against the current spec, but StableEnvironmentController_export is filtered out of the CLI by workflow.yaml while remaining in openapi.stable.json, which this script reads unfiltered.

docs/ is never regenerated — this drove several fixes

An earlier revision of this PR claimed the description non-idempotency "cannot surface in yarn codegen, which regenerates docs/ first." That was false. docs/ is produced by cmd/gendocs, and nothing in the repo invokes it — codegen is speakeasy run && tsx ./scripts/rewrite_docs.ts.

Proof: commit 2868969 is a Speakeasy run that wrote 24 files under internal/cli/ — including the entire test command tree — and zero under docs/. docs/ was last touched 116 commits ago.

So this script is always re-applied to its own previous output, and must be idempotent against it.

What this PR does

Fix Why
Remove the slice; replaceAll; skip x-speakeasy-ignore; skip operations with no x-vf.cli The original defect and its three masked crashes
Delimit injected descriptions with <!-- vf:description --> and replace in place Each run previously appended another copy — measured +1 copy and +33 bytes per run, unbounded. Also handles a changed description, which a "skip if present" guard would not
Replacer functions instead of replacement strings $1, $&, $', $$ are substitution directives. An example containing awk '{print $1}' rewrote $1 to the literal text ### Examples; printf $'\n' spliced the rest of the document into the code fence. Silent, exit 0
Throw when no enriched command has a docs page That state means docs/ was never generated. Warning and exiting 0 turned a broken pipeline into a green build
Rethrow anything that is not ENOENT EACCES and EISDIR were reported as "no docs page", with advice that cannot help
Name the operation on parse failure Across 151 operations a bare ZodError pointed only at rewrite_docs.ts
Reject injected content carrying its own ### heading It would break the section anchors this script and the next run depend on
Warning now says go run ./cmd/gendocs The old text named a command that does not write docs/, and inside yarn codegen had already run moments earlier
Declared OperationExtensions interface instead of Record<string, unknown> A typo in an extension key is now a compile error, not a guard that silently never fires

Read only when there is something to inject; write only when content changed.

Tests

Adds test/rewrite-docs.test.ts — the first coverage of scripts/. Nine hermetic cases driving the real script through execa in a temp directory. Six fail against the previous revision of this branch, including both corruption bugs.

Verification

Toolchain recovered from .yarn/cache, which vendors every pinned package including the private @voiceflow/tsconfig@1.17.0:

  • tsc 7.0.20 errors across the project, identical to the master baseline
  • oxlint 1.71.0 — no correctness violations
  • oxfmt 0.56.0 — clean under the repo's style (printWidth 120, single quotes), validated by confirming existing committed files pass under the same settings
  • vitest — 9/9 pass

Behaviour against the committed .speakeasy/out.openapi.yaml:

Case Result
Real spec, as committed exit 0, zero docs changed, zero files touched
Original script + synthetic enrichment 7 pages
Slice removed only crash — ENOENT docs/vf_variable_create-many.md
Slice removed, ignores skipped, single-space replace crash — ENOENT docs/vf_api-tool_variable list.md
This PR + synthetic enrichment 121 pages, exit 0; runs 2 and 3 byte-identical
$-laden example $1, $$, $' preserved verbatim
Every page missing exit 1, names go run ./cmd/gendocs
Unreadable page (EACCES) exit 1, rethrown
Malformed cli exit 1, Invalid x-vf.cli metadata on PATCH /v1/stable/synthetic

Zero docs churn on the committed spec is the safety proof — this is a pure pipeline fix and touches nothing under docs/.

Not verified

yarn codegen was never run. speakeasy is not installed and openapi.stable.json is gitignored and absent, so validation used the committed transformed spec. That spec excludes StableEnvironmentController_export — problem 3 above — which is therefore handled by construction and synthetic test, not observation.

The repo's own yarn install also fails: @voiceflow/oxlint-config and @voiceflow/oxfmt-config 404 from the public registry. Everything above was run by installing from .yarn/cache with those two removed.

Follow-ups worth filing

  1. docs/ is 116 commits stale and nothing regenerates it. 18 non-ignored commands — the whole vf test family — exist in the spec and in internal/cli/test/ with no docs page. That is why the "121 pages" figure above is a property of the current stale tree; after a real regeneration the reach is 139. codegen should probably be speakeasy run && go run ./cmd/gendocs && tsx ./scripts/rewrite_docs.ts, but that regenerates 153 files and belongs in its own PR.
  2. Write failures are still uncaught, leaving a partially enriched tree. The fail-soft rationale is applied to the read path but not the write path.
  3. Injected examples land unfenced. With zero authored examples there is no convention yet; the first author must supply their own code fences or the page renders as a paragraph. Worth documenting wherever x-vf is authored.

scripts/rewrite_docs.ts runs as the second half of `yarn codegen`. It iterated
`Object.values(document.paths).slice(0, 3)`, so x-vf enrichment reached 7 of 151
operations and was silently skipped for the other 144. The slice was present in
the file's initial commit (163f316), alongside no other limiting logic, so it is
debug residue rather than a deliberate bound.

Removing the slice alone crashes codegen. Three problems were masked by it:

1. `cli.command.replace(' ', '_')` substitutes only the first space, so every
   three-word command produced a path with a literal space --
   `docs/vf_api-tool_variable create.md` -- which does not exist. 69 of 151
   commands are three-word. cmd/gendocs builds the real filename with
   `strings.ReplaceAll(cmd.CommandPath(), " ", "_")`; this now matches it.

2. Operations marked `x-speakeasy-ignore: true` get no command and therefore no
   docs page. All 12 are `create-many` batch endpoints. Iterating them raised
   ENOENT on `docs/vf_variable_create-many.md` and aborted the run.

3. `OperationMetadata.parse(operation['x-vf'])` throws on an operation carrying
   no CLI metadata. Every operation in the current spec has `x-vf.cli`, but
   `StableEnvironmentController_export` is filtered out of the generated CLI
   while remaining in openapi.stable.json, so it is reachable here and untested.
   Operations without `x-vf.cli` are now skipped as "not a CLI command"; a
   malformed `cli` block still throws, because that is a spec error.

A command that carries enrichment but has no docs page is now reported by name
and skipped rather than aborting the run, so a cosmetic docs step cannot leave a
half-generated tree behind. Pages are read only when there is something to
inject and written only when content actually changed.

The commented-out link-rewriting and sidebarTitle blocks are left untouched.
Both were commented in the same initial commit with no recorded rationale, and
neither is currently correct: the link rewrite calls `replaceAll` with a
non-global RegExp, which throws TypeError, and the frontmatter block prepends
unconditionally, so it would stack a new block on every run.

No operation in the spec currently defines `cli.example` or `cli.description`,
so this produces zero changes under docs/ today. Verified against the committed
.speakeasy/out.openapi.yaml: the script reports 0 rewrites and docs/ is
byte-identical. With enrichment injected into all 151 operations it rewrites
121 pages, up from 7.
Copilot AI lite review requested due to automatic review settings August 11, 2026 21:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes scripts/rewrite_docs.ts so OpenAPI x-vf CLI enrichment is applied across all operations (instead of being limited by a debug-era slice), while avoiding hard failures when docs pages aren’t present and minimizing unnecessary rewrites.

Changes:

  • Remove the slice(0, 3) limit so all paths/operations are considered for enrichment.
  • Skip ignored / non-CLI operations and avoid rewriting docs when content is unchanged.
  • Improve filename generation to match cmd/gendocs behavior (ReplaceAll-style space handling) and add end-of-run reporting (rewrites + missing pages).
Suppressed comments (1)

scripts/rewrite_docs.ts:93

  • This warning suggests rerunning speakeasy run, but missing docs pages can also occur when an operation/command is intentionally filtered out (e.g., via workflow transformations). Wording this more generically will avoid misleading users.
  console.warn(
    `[rewrite_docs] ${missingDocsPages.length} command(s) carry x-vf enrichment but have no docs page. ` +
      `Run \`speakeasy run\` to regenerate docs/ first:`
  );

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread scripts/rewrite_docs.ts
Comment on lines +54 to +59
try {
markdown = await fs.readFile(docsPath, 'utf-8');
} catch {
missingDocsPages.push(`${cli.command} -> ${docsPath}`);
continue;
}
Addresses adversarial review of this PR. Two of the findings are corruption bugs
that this PR would have armed rather than caused: it takes the enrichment path
from unreachable to live on 121+ pages, so the multiplier must not land before
the fixes.

A correction first. This PR previously claimed the description non-idempotency
"cannot surface in `yarn codegen`, which regenerates docs/ first". That is false.
`docs/` is produced by cmd/gendocs, and nothing in the repository invokes it --
`codegen` is `speakeasy run && tsx ./scripts/rewrite_docs.ts`. Proof: commit
2868969 is a Speakeasy run that wrote 24 files under internal/cli/, including the
entire `test` command tree, and zero files under docs/. docs/ was last touched
116 commits ago. So this script is always re-applied to its own previous output.

  - Injected descriptions are now delimited by `<!-- vf:description -->` markers
    and replaced in place. Previously each run appended another copy: measured at
    +1 copy and +33 bytes per run, unbounded. Also handles a changed description,
    which a "skip if already present" guard would not.

  - Both injections now use replacer functions instead of replacement strings.
    `$1`, `$&`, `$'` and `$$` are substitution directives inside a replacement
    string, and shell examples contain them routinely. An example containing
    `awk '{print $1}'` previously rewrote `$1` to the literal text `### Examples`,
    and `printf $'\n'` spliced the remainder of the document into the code fence,
    silently and with exit 0.

  - If no enriched command has a docs page at all, the script now throws instead
    of warning. That state means docs/ was never generated, and exiting 0 there
    turned a broken pipeline into a green build.

  - The read guard now rethrows anything that is not ENOENT. EACCES and EISDIR
    were being reported as "no docs page" with advice that could not help.

  - Parse failures name the operation. Across 151 operations a bare ZodError
    pointed only at rewrite_docs.ts. A missing key is the most likely authoring
    error in a hand-maintained extension.

  - Injected content carrying its own `### ` heading is rejected, since it would
    break the section anchors that this script and the next run depend on.

  - The warning now says `go run ./cmd/gendocs`, not `speakeasy run`. The old
    text named a command that does not write docs/ and, inside `yarn codegen`,
    had already run moments earlier.

  - `Record<string, unknown>` replaced with a declared `OperationExtensions`
    interface, so a typo in an extension key is a compile error rather than a
    guard that silently never fires.

Adds test/rewrite-docs.test.ts, the first coverage of scripts/. Nine hermetic
cases driving the real script through execa in a temp directory. Six of them fail
against the previous revision of this branch, including both corruption bugs.

Verified with the repository's own pinned toolchain, recovered from .yarn/cache:
tsc 7.0.2 with @voiceflow/tsconfig 1.17.0 reports 0 errors across the project,
oxlint reports no correctness violations, and oxfmt is clean under the repo's
style. Against the committed spec the script still rewrites 0 pages and leaves
docs/ byte-identical.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants