feat(search): preserve indexing progress and report snapshot coverage

[hlubek]

Whole-project indexing can repeatedly exceed a deadline without retaining useful progress. This change makes an entry version the durable unit of progress while allowing concurrent entries to share provider batches. Search must still distinguish an incomplete index from a complete search with no matches.

Ownership decisions

• SDD owns search truth. The application selects and authorizes home and dependency snapshots, invokes one PrepareSearch(ctx, target) error callback, then derives coverage from published versions and retrieves against those same snapshots. The MCP path uses this application composition. The consumer owns scheduling, retries and waiting policy. Normal wait-budget expiry returns control without asserting completeness; preparation and lazy-iteration failures remain errors.
• Publication owns visibility. An entry becomes complete and searchable together, including an entry with zero chunks. An unfinished entry may repeat embedding after interruption; completed siblings survive. Exact source revision locates reproducible job input, while search freshness asks for a revision containing a write. Those guarantees cannot substitute for each other.
• Provider batches are transport units. A shared batcher bounds admission and dispatch across callers. Entries larger than the buffer must make progress, and one caller's cancellation must not cancel another's work. Existing routing, deadline and observation decorators own query separation, provider timeouts and once-per-call usage.

A consumer's existing mutation finalizer and graph-write/recovery protocol remain the starting point for durable write-triggered scheduling. Queue-state APIs, a second prepare/execute search API, within-entry checkpoints, cross-process batching and an event subsystem are outside this change.

Write-triggered and whole-project discovery share OSS eligibility, hashing and descriptor resolution. DiscoverSearchEntriesQuery.EntryIDs is nil for all entries and otherwise a nonempty set of valid full IDs. Cursors bind the canonical selection as well as revision and index configuration. Valid absent or ineligible entries produce no requirement; selected unreadable documents and source failures remain errors.

AppliedMutation.AffectedEntryIDs resolves document changes, deletions and attachment owners without asking consumers to parse SDD paths. It supplies only a selection. A consumer must establish durable exact-source retention before enqueueing discovery, potentially using a finalized Git revision instead of an earlier workspace revision. Mutations affecting no entries create no discovery job.

Review considerations

Start with the composition examples, then review target preparation and coverage and publication visibility. The package Go documentation and mutation finalizer example describe consumer adoption. The approved graph plan records the reasoning; its done signal is deliberately deferred until after the first review round. The branch also includes an existing conceptual event-stream signal from its local base; no event-stream implementation is included.

Compatibility and cost choices worth examining:

• Derivation schema is part of the full entry-state hash. Existing rows remain stored, but current entries require fresh publication. Embedding configuration changes must also change the fingerprint.
• Branch-specific pinned reads carry the authorized branch. Local filesystem adapters declare their branch scope and reject mismatches.
• All texts pass byte/unit validation before any request item is admitted to shared batching; valid large requests still enter incrementally.
• Custom preparation requires pinned snapshot reads and publication-aware indexing for every selected project. Legacy adapters retain synchronous/no-sync search without coverage metadata.
• Coverage hashes eligible target entries and attachments, including with synchronization disabled. Local snapshot leases retain immutable bytes in memory and do not provide historical source retention across process restart.
• Cached manifest reads and a cached published-only retrieval view avoid repeated decoding and allocating every candidate result. Manifest rewriting and local collection reloads remain per-entry costs.
• Concurrency and buffering limits are explicit and provisional. A real local provider exercise showed variable query latency; consumer workload measurements must determine production settings.

Validation

devbox run test passed for the root and nested example modules; vet, the lint wrapper, targeted race checks and build/view smoke checks passed. Lint retains 50 existing test-package warnings.

Application and MCP tests cover local and external-consumer composition, fixed snapshots, causal freshness, explicit failures and incomplete versus complete zero-match results. Publication and batcher tests cover zero chunks, retries, cancellation, shutdown, oversized admission and unpublished-row exclusion.

A real Ollama exercise killed the indexing process after two publications. Restart preserved and skipped those entries, completed healthy siblings despite one injected failure, and eventually published all 17 entries. A final attempt performed no document embedding. Provider measurements, cost benchmarks and reproduction commands are staged as a graph attachment for the deferred completion capture; they are no longer kept as package Markdown.

Review-extension regressions cover selection normalization and cursor compatibility, unreadable versus absent entries, skipped attachment reads, attachment-only/deleted entry mapping, finalized-source scheduling, branch preservation through preparation/coverage/retrieval, and zero provider calls for invalid trailing input. The two package Markdown files are removed; API guidance lives in Go docs and examples.