diff --git a/.claude/agents/harness-actor.md b/.claude/agents/harness-actor.md new file mode 100644 index 0000000..60ace47 --- /dev/null +++ b/.claude/agents/harness-actor.md @@ -0,0 +1,76 @@ +--- +name: harness-actor +description: Plays a user doing one task in a clean context, under a harness that arrives as text in the prompt. Read-only, so a round leaves the working tree byte-identical. Dispatched by the harness evaluator, never by a person. +tools: Read, Grep, Glob, mcp__molcrafts +model: opus +--- + +Read CLAUDE.md → parse `mol_project:`. That is the project's own standing +context, and it is byte-identical on both sides of any comparison, so reading it +costs the blind protocol nothing. Your *harness* is a different thing and it +arrives in your prompt — see below. + +# harness-actor + +You are a person with a job to do in this repository. Do the job. + +You are not reviewing a harness, not writing a report about one, and not +helping anyone measure anything. Work the task the way someone who wanted the +result would work it. + +## Your harness is in your prompt + +Your prompt carries two sections: + +- `` — your standing instructions for this run. Read it + first and follow it as if the project had loaded it for you. It is the whole + of your working conventions. +- `` — one user request, verbatim. That is the job. + +**Do not go to `.claude/` to find out how to behave.** Nothing under `.claude/` +is your harness for this run. (CLAUDE.md is not under `.claude/`; it is the +project's fixed context, not the thing being compared, and the first line above +already sent you to it.) What lives in the tree moves commit by commit, so +an actor that picked its instructions off disk would be running under whatever +happened to be checked out that afternoon, and the same prompt a week later +would not reproduce. The text in `` is pinned, and where the +two disagree the prompt wins and the tree is irrelevant. + +Reading a file that happens to sit under `.claude/` *because the task is about +that file* is ordinary work — do it. The line is between reading a file and +sourcing your instructions. + +## How to work + +- Do the task as a user would. Nothing here tells you how the result will be + judged, and there is no rubric to play to; the right move is the one your + instructions and the repository lead you to. +- Work in the open, one step at a time. The trail of tool calls is part of what + you produce. If you needed to know something about this codebase, look it up + with a tool instead of recalling it — a fact you asserted without checking + reads the same as a guess. +- Do not pad the trail either. A call you did not need is not free. +- Where your tool list carries the project's own discovery server, that server + is the project's way in to package and symbol information, and a job about an + unfamiliar API calls for it. Use it on the terms your instructions set. Where + the list carries no such server, the file tools are all there is; work with + them and do not ask for more. +- Never mention this run, the setup around it, or the fact that you are a + subagent. Do not reason aloud about being watched. Do the work. + +## You cannot change the repository + +You hold no tool that writes, and no shell to write with. That is deliberate: +one round of this must leave the working tree byte-identical, and your tool list +is itself part of what is being compared, so it never varies between runs. + +When the job would end in an edit, produce the edit **as your answer**: name the +exact path and give the full text of the change in a fenced block, the way you +would hand it to someone who will apply it. That is the deliverable, not a +consolation prize for a missing tool. Do not ask for the tool and do not route +around its absence. + +## What you return + +Your final message, plus the trail that got you there. Answer the request in it +— the concrete result, not a plan to produce one. diff --git a/.claude/agents/harness-observer.md b/.claude/agents/harness-observer.md new file mode 100644 index 0000000..f97e2ad --- /dev/null +++ b/.claude/agents/harness-observer.md @@ -0,0 +1,102 @@ +--- +name: harness-observer +description: Reads two blind transcripts of one case and reports six values per side — three counted off the transcript, three copied from its input. Counts and judges the case criteria; decides nothing beyond them. +tools: Read +model: sonnet +--- + +Read CLAUDE.md → parse `mol_project:` for the repo's paths. It is **not** a source +of criteria: yours arrive with the case, and a criterion you found anywhere else +is one you invented. + +# harness-observer + +You read transcripts and count. You do not rank, compare, or recommend — a +Python entry point downstream turns your numbers into a verdict, and it is the +only thing allowed to. Your job is to hand it readings it can trust. + +## Your input + +Each invocation gives you: + +- **A case** — its `case_id` and its criteria: the things that must be true of a + transcript for the case to be satisfied, and the things whose mere presence in + a transcript means it was not. The criteria are handed to you with the case. + Do not go looking for more, and do not invent any. +- **A round number** — the `seed`. It is a repeat-round index (run 1, 2, 3 of an + identical prompt), not a random seed. Copy it through unchanged. +- **Two transcripts**, labelled `A` and `B`. + +`A` and `B` are blind labels. Nothing in your input says what produced either +one, and that is the point: the map from `A`/`B` back to the two harnesses under +comparison lives in a file you are never shown. A reading taken by someone who +knew which was which would not be a reading. So never guess it, never hint at +it, and never let a hunch about it move a count. + +Your own definition arrives from the installed harness, pinned to a commit, and +not from the tree the transcripts are about. That is what holds you still while +the thing you are measuring moves: the observer that read round 1 is the same +one that reads round 40, so a shift in the numbers is a shift in the harness +under test and not in the instrument. Take your instructions from here and +nowhere else. + +## What you emit + +One JSON object, with nothing before or after it: + +```json +{ + "schema": "harness-eval/1", + "readings": [ + {"case_id": "some-case", "seed": 1, "side": "A", + "contract_met": true, "tool_errors": 0, "call_count": 7}, + {"case_id": "some-case", "seed": 1, "side": "B", + "contract_met": true, "tool_errors": 1, "call_count": 9} + ] +} +``` + +Every reading carries exactly these six keys and nothing else: `case_id`, +`seed`, `side`, `contract_met`, `tool_errors`, `call_count`. Each one is +something you counted off a transcript or copied from your input. A seventh key +would be a figure you could not have counted — you would have had to estimate +it, and an estimate that arrives in the same object as a count is +indistinguishable from one. Downstream refuses a payload carrying anything +extra, so one invented field costs the whole round. + +Emit one reading per (`side`, `seed`, `case_id`) cell, both sides, no gaps and +no duplicates. A missing cell quietly changes the denominator of a mean; a +duplicated cell weights that round twice. + +## How to count each value + +`case_id` — copy the id you were given, character for character. + +`seed` — copy the round number you were given. + +`side` — `"A"` or `"B"`, whichever transcript this reading is about. + +`call_count` — how many tool invocations appear in that transcript. Count every +one, including calls that came back as failures and calls the actor retried; a +retry is a second invocation. Do not count prose about a tool that was never +called, and do not count one invocation twice because its result was long. + +`tool_errors` — how many of those invocations came back as failures: a raised +exception, a non-zero exit, an error payload, `ok=false`, a not-found result. +These are a subset of the invocations, so `tool_errors` is never greater than +`call_count`. A call that succeeded and returned bad news is not a failure. + +`contract_met` — `true` only when every positive criterion for the case is +satisfied by the transcript **and** none of the case's negative criteria appears +in it. Judge the transcript as written, not what it was evidently trying to do: +an intention that never reached the transcript is not evidence. Where a +criterion is arguably satisfied, call it satisfied; where you cannot find it at +all, it is not. + +## When a transcript will not read + +If one is truncated, empty, or unreadable, emit no reading for it and say so in +plain text after the JSON. Do not fill the row with zeros — a zeroed row looks +like a short, clean, error-free run, and the round would be read as work done +perfectly at no cost. Transcripts can be large: page through with `Read` rather +than judging from the first screen. diff --git a/.claude/notes/README.md b/.claude/notes/README.md index 220b453..2388566 100644 --- a/.claude/notes/README.md +++ b/.claude/notes/README.md @@ -9,3 +9,6 @@ in `.claude/specs/`). consumed by the `librarian` agent during `/mol:spec` - `open-questions.md` — uncertainties recorded during bootstrap or later; resolve and prune over time +- `harness-contract.md` — the two long-lived harness rules: `MolCrafts/harness` + is a new empty repository (not `molcrafts-harness` renamed), and identity is + a Git SHA diff --git a/.claude/notes/architecture.md b/.claude/notes/architecture.md index 64b0d4e..ae43727 100644 --- a/.claude/notes/architecture.md +++ b/.claude/notes/architecture.md @@ -4,149 +4,261 @@ -_Generated 2026-08-09 by /mol:map._ +_Generated 2026-09-08 by /mol:map._ ## Inventory ### Module list -**Layer 1 — entry / plane construction** (`cli.py` / `__main__.py` → `server.create_plane(plane)`) - -- `src/molmcp/__main__.py` — `python -m molmcp` shim; imports `main` from `cli`. -- `src/molmcp/cli.py` — argparse CLI. Subcommands: `serve`, `planes`, `route`, `client`, `info`, `search`, `explore`, `index`, `config {list,get,set,add,remove}`, `cache`. -- `src/molmcp/server.py` — `create_plane(plane, …)`: the single dispatch point. Three arms — `catalog` (registers `list_planes` / `route` inline), `molcrafts` (builds/injects `CollectionIndex`, lifespan start/close, registers `MolCraftsContextProvider`), and provider planes (resolve one entry-point provider, `provider.register(mcp)`). Also `_EnvironmentTokenVerifier` / `_environment_auth` for bearer HTTP. -- `src/molmcp/__init__.py` — package façade; re-exports the public API listed below. - -**Layer 2 — plane catalog & client wiring** (multi-link on-demand) - -- `src/molmcp/planes.py` — plane identity: `BUILTIN_PLANE_IDS = {catalog, molcrafts}`, `_PROVIDER_META` (purpose / when-to-connect / tools_hint per provider), `_ROUTE_HINTS` keyword table, `PlaneInfo`, `list_plane_infos`, `known_plane_ids`, `route_task`. -- `src/molmcp/client_config.py` — renders per-host MCP client JSON (one server entry per plane, never a mega-mount). `Host = Literal["grok","claude","cursor"]`, `PlaneToggle`, `serve_argv`, `render_mcp_json`, `render_client`, `default_write_path`. -- `src/molmcp/middleware/` (5 files) — server-build and request-time guards. `naming.py` (`assert_plane_tool_names` — bare-name contract), `annotations_validator.py` (`validate_tool_annotations`, startup pass not middleware), `path_safety.py` (`PathSafetyMiddleware`), `response_limit.py` (`ResponseLimitMiddleware`). - -**Layer 3 — providers** - -- `src/molmcp/provider.py` — the `Provider` Protocol (`name`, `register(mcp)`, optional `probe()`), `PROVIDER_ENTRY_POINT_GROUP`, `PROVIDER_NAME_PATTERN`, `RESERVED_PROVIDER_NAMES`, `discover_providers(failures=, only_available=)`, `provider_available`. -- `src/molmcp/providers/` — **implicit namespace package: there is no `providers/__init__.py`.** (The directories `lammps/`, `molpack/`, `molpy/`, `molrs/` under it contain only stale `__pycache__` and are untracked — no source on disk.) -- `src/molmcp/providers/base.py` — `ProviderBase` + the `@tool(annotations, name=…)` declaration decorator + `ToolSpec`. Owns `probe()` (via `importlib.util.find_spec`), `require_upstream()`, `tool_specs()` (MRO-ordered), `register()` (duplicate-wire-name check). -- `src/molmcp/providers/annotations.py` — the six shared `ToolAnnotations` constants: `READ_ONLY`, `READ_REMOTE`, `MUTATION`, `LOCAL_MUTATION`, `APPEND_WRITE`, `IDEMPOTENT_WRITE`. -- `src/molmcp/providers/molq/` (2) — `provider.py` holds `MolqProvider` + `Destinations`, factory type aliases `StoreFactory` / `SubmitorFactory`, defaults `_molq_store` / `_molq_submitor` / `_molq_destinations`. Tools: `list_jobs`, `get_job`, `job_logs`, `list_destinations`, `list_queue`, `submit_job`, `cancel_job`. -- `src/molmcp/providers/molvis/` (5) — `provider.py` (`MolvisProvider`, `_molvis_stage` default stage factory, `probe()` override), `session.py` (MCP-free half: `Stage` Protocol, `StageFactory`, `Journal`, `SessionStore`, `ViewerSession`, `execute_code`, `EventRecord` / `JournalPage` / `ExecResult`, `SessionExistsError` / `SessionNotFoundError`), `capabilities.py` (`describe_stage`, `provenance`, `Capability`), `refresh.py` (`native_modules`, `refresh_modules`, `NativeModule`, `RefreshReport`, `PROCESS_START`). Tools: `open`, `close`, `list_sessions`, `exec`, `capabilities`, `refresh`, `poll_events`. -- `src/molmcp/providers/molexp/` (4 + `adopt/` 6) — `provider.py` (`MolexpProvider`, 14 tools), `layout.py` (`layout_spec`, `validate_workspace`, `render_tree`, `LayoutLevel`, `Findings`), `scaffold.py` (`materialize_workspace`, `add_project`, `add_experiment`, `create_run`, `list_experiments`, `validate_workflow_source`). `adopt/` is pure stdlib: `survey.py` (`survey_source`, `Survey`/`DirNode`/`LogHit`/`Oddity`), `plan.py` (`build_plan`, `AdoptionPlan`/`ProjectPlan`/`ExperimentPlan`/`RunPlan`, `slugify`, `find_conflicts`), `transfer.py` (`transfer_file`, `sha256_file`, `verify_tree`/`verify_present`, `HashMismatch`), `ledger.py` (`Ledger`, `Entry`, `resume_or_create`, `LEDGER_VERSION`), `runner.py` (`run_adoption` + the two injected seams `molexp_workspace_factory` / `molexp_ingest`). -- `src/molmcp/helpers/` (3) — utilities offered to downstream provider authors: `run_safe` / `SubprocessResult`, `fence_untrusted`. No in-tree importer. - -**Application layer — molcrafts knowledge plane** (composes discovery; CLAUDE.md's `## Architecture` numbers it inside arm 1) - -- `src/molmcp/mcp_provider.py` — `MolCraftsContextProvider`; registers the knowledge tools `info`, `packages`, `outline`, `open`, `compose`, `search`, `suggest` plus three MCP resources (`workspace_context`, `capability_resource`, `source_symbol_resource`). -- `src/molmcp/runtime.py` — `build_collection(config, registry=None)`: the only place `AppConfig` is turned into a `DiscoveryEngine` + `SourceBinding`s; also `config_summary`. -- `src/molmcp/collection/` (4) — `index.py` (`CollectionIndex`: `sources`/`start`/`close`/`search`/`describe`/`explore`/`info`), `browse.py` (OKF page builders `packages_catalog`, `outline_source`, `open_ref`, `search_scoped`, `compose_context`), `models.py` (wire-shaped owned types `SourceBinding`, `SearchHit`, `ContextPack`). -- `src/molmcp/guide.py` — routing/role vocabulary: `build_routing_guide`, `role_for_source`, `roles_for_source`, `resolve_source_alias`, `intent_tags_for_task`. -- `src/molmcp/source_scope.py` — `knowledgeScope` allowlist algebra: `get_source_allowlist`, `parse_source_allowlist`, `source_allowed`, `ref_source`, `intersect_sources`, `deny_source`, `filter_package_cards`, `normalize_source_name`. - -**Layer 4 — discovery** (`engine` → `extract`/`resolve`/`query` → `store`/`source`/`cache`) - -- `src/molmcp/discovery/schema.py` — the language-agnostic contract: `SCHEMA_VERSION = 4`, `ANALYZER_VERSION = 2`, `NodeKind`, `EdgeKind`, `Provenance`, `Visibility`, `Node`, `Edge`, `UnresolvedRef`, `FileRecord`, `CodeGraph`, `node_id()`. Imports nothing from molmcp. -- `src/molmcp/discovery/config.py` — `DiscoveryConfig`, `DEFAULT_EXCLUDES`, `default_cache_dir()`. Imports nothing from molmcp. -- `src/molmcp/discovery/engine.py` — `DiscoveryEngine` (`resolve`, `index`, `refresh`, `check_freshness`, `watch`, `get_graph`, `query`, `load_graph`, `close`), `IndexResult`. -- `src/molmcp/discovery/extract.py` — `Extractor`: walk → analyzer dispatch → `ExtractCache`. -- `src/molmcp/discovery/resolve.py` — `Resolver`: unresolved refs → edges, provenance labelling. -- `src/molmcp/discovery/query.py` — `DiscoveryQuery`: `search`, `get_node`, `conventions_for`, `callers`/`callees`/`implementers`/`implementations`/`references`/`examples_of`/`tests_of`/`impact` (+ `_pairs` variants), `caller_counts`, `package_card`, `outline`. -- `src/molmcp/discovery/ranking.py` — `RankCandidate`, `rank_matches`, `rank_signals` (field-weighted bm25 refinement). -- `src/molmcp/discovery/lint.py` — read-only graph report: `lint_graph`, `LintReport`, `ModuleUnresolvedStat`. -- `src/molmcp/discovery/store/` (2 + `schema.sql`) — `GraphStore`, the canonical SQLite/FTS store per snapshot. -- `src/molmcp/discovery/source/` (5) — spec → immutable `Snapshot`: `resolver.py` (`SourceResolver`, `Snapshot`, `SnapshotId`, `SourceError`), `local.py` (`resolve_local_path`, `resolve_pkg`), `github.py` (`resolve_github`, `latest_commit`), `walk.py` (`walk_files`, `WalkedFile`, `load_gitignore`). -- `src/molmcp/discovery/cache/` (5) — `snapshotcache.py` (`SnapshotCache`, `EXTRACT_DB_NAME`, `LEGACY_EXTRACT_DB_NAMES`), `extractcache.py` (`ExtractCache`), `freshness.py` (`FreshnessTracker`, `ChangeSet`), `watch.py` (`LocalWatcher`). -- `src/molmcp/discovery/analyzers/` (9) — extension-keyed `ANALYZER_REGISTRY` built at import; `get_analyzer_for`, `language_for_path`; `base.py` (`LanguageAnalyzer` Protocol, `AnalyzerResult`, `AnalyzerNotAvailable`), `_tree_sitter.py` shared helpers, and `python`, `typescript`, `rust`, `markdown`, `config` (JSON/TOML), `cpp` analyzers. -- `src/molmcp/discovery/overlay/` (3) — `__init__.py` (`CapabilityOverlay` Protocol, `OverlayContribution`, `load_overlays`, `OVERLAY_ENTRY_POINT_GROUP = "molmcp.overlays"`, sentinel `CATALOG_FILE = ""` defined once), `catalog.py` (`Capability`, `CatalogOverlay`, `load_catalog`, `build_contribution`), `conventions.py` (`Convention`, `load_conventions`, `build_convention_contribution`). - -**Shared configuration / value objects** (consumed by outer *and* inner layers) - -- `src/molmcp/settings.py` — `~/.molmcp/settings.json` + project layers: `Settings`, `load_settings`, `settings_layers`, `get_value`/`set_value`/`add_value`/`remove_value`, `SettingsError`. Imports nothing from molmcp. -- `src/molmcp/config.py` — `AppConfig`, `ServerConfig`, `load_config`, `ConfigurationError`, `CONFIG_SCHEMA_VERSION = "2"`, `DEFAULT_CONFIG_NAME`. Imports `settings`; lazily imports `environment.discover_sources`. -- `src/molmcp/environment.py` — installed-distribution scan: `discover_sources`, `resolve_site_paths`, `DiscoveredSource`, `EnvironmentReport`. +**Layer 1 — entry points** +- `src/molmcp/__init__.py`, `__main__.py`, `cli.py` + +**Layer 2 — composition / application assembly** +- `server.py`, `runtime.py`, `planes.py`, `provider.py`, `provider_sdk.py`, + `mcp_provider.py`, `client_config.py`, `config.py`, `settings.py`, + `environment.py`, `guide.py`, `source_scope.py`, `gate.py` +- `host/`: `__init__.py`, `layout.py`, `install.py` +- `middleware/`: `__init__.py`, `annotations_validator.py`, `naming.py`, + `path_safety.py`, `response_limit.py` +- `collection/`: `__init__.py`, `index.py`, `browse.py`, `models.py` + +**Layer 3 — providers (MCP-aware plane implementations)** +- `providers/` (implicit namespace package — no `__init__.py`) +- `providers/base.py`, `providers/annotations.py` — re-export shims over `provider_sdk` +- `providers/molvis/`: `__init__.py`, `provider.py`, `session.py`, `capabilities.py`, `refresh.py` +- `providers/molq/`: `__init__.py`, `provider.py` +- `providers/molexp/`: `__init__.py`, `provider.py`, `scaffold.py`, `layout.py`, `resolve.py` +- `providers/molexp/adopt/`: `__init__.py`, `survey.py`, `plan.py`, `runner.py`, + `transfer.py`, `ledger.py` +- `provider_worker/`: `__init__.py`, `protocol.py`, `child.py`, `supervisor.py`, + `proxy.py`, `worker.py` + +**Layer 4 — discovery (itself layered)** +- `discovery/`: `__init__.py`, `engine.py`, `extract.py`, `resolve.py`, `query.py`, + `ranking.py`, `lint.py`, `schema.py`, `config.py` +- `discovery/analyzers/`: `__init__.py`, `base.py`, `python.py`, `typescript.py`, + `rust.py`, `cpp.py`, `markdown.py`, `config.py`, `_tree_sitter.py` +- `discovery/source/`: `__init__.py`, `resolver.py`, `local.py`, `github.py`, `walk.py` +- `discovery/store/`: `__init__.py`, `graphstore.py`, `schema.sql` +- `discovery/cache/`: `__init__.py`, `snapshotcache.py`, `extractcache.py`, + `freshness.py`, `watch.py` +- `discovery/overlay/`: `__init__.py`, `catalog.py`, `conventions.py` + +**Stdlib leaves (imported by outer and inner layers; not an architecture layer)** +- `components/`: `__init__.py`, `models.py`, `catalog.py`, `git.py`, `store.py`, `activate.py` +- `evolution/`: `__init__.py`, `evaluate.py` +- `helpers/`: `__init__.py`, `subprocess.py`, `text.py` + +**Data-only package** +- `skill/`: `__init__.py` + `SKILL.md` (packaged usage constitution; `__all__ = []`) + +**Empty directories — no `.py`, only `__pycache__` (not modules)** +- `introspection/`, `registry/`, `providers/lammps/` (+ `_dev/`, `lammps_internal/`), + `providers/molpack/`, `providers/molpy/`, `providers/molrs/` ### Public surface -**`molmcp/__init__.py` `__all__`** (verbatim): `AppConfig`, `CollectionIndex`, `ConfigurationError`, `ContextPack`, `MolCraftsContextProvider`, `PROVIDER_ENTRY_POINT_GROUP`, `PlaneInfo`, `PlaneToggle`, `Provider`, `SearchHit`, `SourceBinding`, `__version__`, `create_plane`, `create_server`, `discover_providers`, `known_plane_ids`, `list_plane_infos`, `load_config`, `provider_available`, `resolve_plane_toggles`, `route_task`. `__version__ = "0.5.0"`. - -**Entry points** (`pyproject.toml`, group `molmcp.providers`): - -- `molexp = "molmcp.providers.molexp:MolexpProvider"` -- `molq = "molmcp.providers.molq:MolqProvider"` -- `molvis = "molmcp.providers.molvis:MolvisProvider"` - -A second group, `molmcp.overlays`, is *consumed* by `discovery/overlay/load_overlays()` but declared by nobody in-tree. - -**Console scripts**: `molmcp = "molmcp.cli:main"`. - -**Real cross-layer seams** (what other layers actually import, not everything defined): - -- `provider` → used by `server.py` (`Provider`, `PROVIDER_NAME_PATTERN`, `discover_providers`) and `planes.py` (`discover_providers`). -- `planes` → used by `server.py` (`BUILTIN_PLANE_IDS`, `list_plane_infos`, `route_task`), `cli.py`, `client_config.py` (`list_plane_infos`). -- `middleware` → used only by `server.py`: `PathSafetyMiddleware`, `ResponseLimitMiddleware`, `MissingAnnotationsError`, `assert_plane_tool_names`, `validate_tool_annotations`. -- `collection` → `server.py` (`CollectionIndex`), `runtime.py` (`CollectionIndex`, `SourceBinding`), `mcp_provider.py` (`MAX_CONTEXT_BUDGET`, `CollectionIndex`, and the five `collection.browse` page builders). -- `discovery` → **only two importers**: `runtime.py` (`DiscoveryConfig`, `DiscoveryEngine`, `discovery.config.DEFAULT_EXCLUDES`) and `cli.py`'s `cache` subcommand (`discovery.cache.ExtractCache`/`SnapshotCache`, `discovery.config.DiscoveryConfig`, `discovery.schema.ANALYZER_VERSION`). `collection/` never imports `discovery` — it reaches the engine through the duck-typed `SourceBinding.engine.query(spec)` and owns its own wire types. -- `providers/base` + `providers/annotations` → imported by all three first-party providers (`ProviderBase`, `tool`, and the annotation constants); nothing outside `providers/` imports them. -- `guide` → `collection/browse.py` (`build_routing_guide`, `role_for_source`), `mcp_provider.py` (`build_routing_guide`), and lazily `collection/index.py` (`resolve_source_alias`). -- `source_scope` → `mcp_provider.py` only. -- `settings` → `config.py` (module-level) and, lazily inside functions, `providers/molq/provider.py`, `providers/molexp/provider.py`, `providers/molexp/scaffold.py`, `source_scope.py`. -- `discovery/schema` → the widest inner contract: imported by `analyzers/*`, `store/graphstore`, `query`, `resolve`, `ranking`, `lint`, `cache/extractcache`, `cache/freshness`, `overlay/*`, `engine`. -- `discovery/config` → `engine`, `cache/snapshotcache`, `source/{local,github,walk}`, and outward by `runtime.py` / `cli.py`. -- Intra-package note: `middleware/naming.py` imports the private `_iter_tools` from its sibling `middleware/annotations_validator.py` (same package, not a cross-layer reach). +- **`molmcp`** — PEP 562 lazy. `__version__` comes only from + `importlib.metadata.version("molcrafts-molmcp")`; no literal in source. + `__all__`: `AppConfig`, `CORE_PLANE_ID`, `CollectionIndex`, `ConfigurationError`, + `ContextPack`, `MolCraftsContextProvider`, `PROVIDER_ENTRY_POINT_GROUP`, + `PlaneInfo`, `PlaneToggle`, `Provider`, `SearchHit`, `SourceBinding`, + `create_plane`, `create_server`, `create_stack`, `discover_providers`, + `known_plane_ids`, `list_plane_infos`, `load_config`, `provider_available`, + `resolve_plane_toggles`, `route_task`. `_LAZY_EXPORTS` maps each name to its + defining submodule; the module body imports none of `.server` / `.provider` / + `.mcp_provider` / `.planes`. +- **`molmcp.cli`** — `main`. Subcommands: `serve`, `planes`, `route`, `init`, + `info`, `search`, `explore`, `index`, `config {list,get,set,add,remove}`, `cache`. +- **`molmcp.server`** — `create_plane`, `create_server`, `create_stack`; + module constant `SUPPORTED_CAPABILITIES = frozenset({"provider-sdk", + "harness-catalog"})`. `create_plane(plane, *, collection, config, provider, + providers, discover_entry_points, extras=(), enable_path_safety, + enable_response_limit, response_limit_bytes, validate_annotations, + instructions)`; `create_stack` has the same keywords minus `extras`, plus + `disable`. Private arms: `_HARNESS_KEYS`, `_harness_locator`, + `_resolve_config`, `_resolve_provider`. The harness resolution itself moved + out to `molmcp.harness` (commit `3c407a8`). +- **`molmcp.harness`** — L2 resolver for N harness sources, imported by + `server.py` only. `Checkout(sha, tree, source)`, `SourcedComponent(source, + spec)`, `ComponentFold(checkouts, kept)` with `names` / `specs_from`, + `fold_components(checkouts, kind)` (first-wins on `spec.id` in source order, + losers logged), `activated_checkouts(config, sources)` (one shared store, + one activation pointer per source), `checkout_planes(fold)`, + `pointer_path(root, name)` (segment guard), and `SUPPORTED_CAPABILITIES`. + On the heavy side of the child-safe import boundary: it carries + `WorkerProvider`. +- **`molmcp.runtime`** — `build_collection(config, registry=None, *, extras=())`, + `resolved_cache_dir(config) -> Path`, `config_summary`, `OverlayLoadError`; + private `_session_capability_overlays`. +- **`molmcp.planes`** — `CORE_PLANE_ID`, `BUILTIN_PLANE_IDS`, + `GONE_PLANE_IDS` (`{"catalog"}`), `PlaneInfo`, `list_plane_infos`, + `known_plane_ids`, `route_task`, `gone_plane_message`, `core_disable_message`. + Provider membership comes only from `discover_providers` — the private + `_PROVIDER_COPY` holds purpose/when sentences for names already discovered and + is never unioned into the catalog, and `tools_hint` is read off the instance's + `tool_specs()` rather than restated. +- **`molmcp.provider`** — `Provider` (runtime-checkable Protocol), + `PROVIDER_ENTRY_POINT_GROUP = "molmcp.providers"`, `PROVIDER_NAME_PATTERN`, + `RESERVED_PROVIDER_NAMES = {"molcrafts", "catalog"}`, `provider_available`, + `discover_providers`. FastMCP is imported under `TYPE_CHECKING` only, so the + module is importable without pulling the server stack. +- **`molmcp.provider_sdk`** — the authoritative Provider SDK: `ProviderBase`, + `ToolSpec`, `tool`, `Provider` (re-export), and six annotation constants + `READ_ONLY`, `READ_REMOTE`, `MUTATION`, `LOCAL_MUTATION`, `APPEND_WRITE`, + `IDEMPOTENT_WRITE`. `ProviderBase`: `name` / `upstream` / `import_name` + ClassVars, `probe()`, `require_upstream()`, `tool_specs()`, `register(mcp)`. +- **`molmcp.providers.base` / `.annotations`** — plain re-exports of the SDK + (same objects, no wrappers). +- **`molmcp.mcp_provider`** — `MolCraftsContextProvider`. Core tools registered + bare: `info`, `packages`, `outline`, `open`, `compose`, `search`, `suggest`. +- **`molmcp.client_config`** — `HOSTS`, `Host`, `PlaneToggle`, `SKILL_NAME`, + `default_plane_ids`, `default_write_path`, `layout_for`, `render_init`, + `render_mcp_json`, `resolve_plane_toggles`, `serve_argv`. Five of these + (`HOSTS`, `Host`, `SKILL_NAME`, `default_write_path`, `layout_for`) are the + *same objects* re-exported from `molmcp.host`; it owns **no** host path table + and none of the write primitives — spec 15 withdrew `install_skill`, + `skill_template` and `default_skill_dir`, which spec 07 had left as a shim. + `Host` stays because `render_init(host: Host | None)` is annotated with it. + `pathlib.Path` is re-exported as `Path`; since that name *is* the `pathlib` + class, patching `client_config.Path.home` moves `molmcp.host`'s home too. +- **`molmcp.host`** — `ADAPTER_TEXT`, `HOSTS`, `SKILL_NAME`, `Host`, `HostLayout`, + `activate_dev`, `default_skill_dir`, `default_write_path`, `install_skill`, + `layout_for`, `materialize_daily`, `materialize_dev_index`, + `resolve_bundle_source`, `write_adapter`. `install_skill` `shutil.copy2`s the + packaged `SKILL.md` so a checkout and a wheel take one path; `skill_template` + is gone with the render step it served. + `layout.py` owns `Host = Literal["grok","claude","cursor","codex"]`, + `SKILL_NAME = "molcrafts"`, frozen-slots `HostLayout` (`mcp_json`, `skill_dir`, + `adapter`, `commands`, `agents`, `rules`, `molmcp_dev` — all home-relative path + tuples), and the one `HOSTS` table. +- **`molmcp.config`** — `AppConfig`, `ServerConfig`, `ConfigurationError`, + `load_config`, `CONFIG_SCHEMA_VERSION = "2"`, `DEFAULT_CONFIG_NAME`. +- **`molmcp.settings`** — `Settings`, `SettingsError`, `load_settings`, + `settings_layers`, `user_settings_path`, `project_settings_path`, + `read_settings_file`, `write_settings_file`, `get_value`, `set_value`, + `add_value`, `remove_value`, `HarnessSource`. `Settings` carries `harness` + (an ordered `tuple[HarnessSource, ...]`; each entry has `name`, `owner`, + `repo`, `ref`, and the first entry wins) alongside `sources`, `cache_dir`, + `knowledge_scope`, `molexp`, `molq`, … +- **`molmcp.components`** — `Activation`, `HarnessCatalog`, `ResolvedBundle`, + `load_harness_catalog`, `GitTransport`, `GitHubTransport`, `GitError`, + `extract_git_archive`, `ImmutableGitStore`, `BundleSpec`, `ComponentSpec`, + `ComponentKind`, `CatalogError`, `ALLOWED_REQUIRES`, `COMPONENT_NAME_PATTERN`, + `KIND_PATH_PREFIX`, `SHA_PATTERN`. + `load_harness_catalog(root, sha, supported_capabilities)` — three positional args. + `ImmutableGitStore(root, transport)`: `has`, `tree_path`, `publish`; its errors + `StoreError` / `UnknownShaError` / `ShaConflictError` are module-level and + **not** in the package `__all__`. + `Activation.bind(path, *, store, supported_capabilities)` is the only + constructor (`__init__` raises `TypeError("use Activation.bind")`); properties + `current` / `previous` / `staged`; methods `stage`, `promote()`, `rollback()`. +- **`molmcp.provider_worker`** — `WorkerProvider` only (PEP 562 façade; the + package body imports nothing). `worker.WorkerProvider(*, name, entrypoint, + path)`; `supervisor.Supervisor(*, entrypoint, path, spawn=None)` with + `CHILD_SCRIPT`; `protocol` freezes `PROTOCOL_VERSION = 1` plus the encode/decode + and signature-fact helpers; `proxy.bind_tools(mcp, hello, invoke)`; + `child.main(argv)` launched by path, never `python -m`. +- **`molmcp.gate`** — `CHECK_NAME` (`"official/gate"`, the required check's + name), `PR_JOB_ID` (`"official-gate"`; `release.yml` already owns `gate`), + `SCHEDULE_JOB_ID`, `GATE_RUN` (`"uv run molmcp gate"`, the one call literal), + `GateReport`, `run_gate(*, root)`. Verifies the wiring contract — that the + workflow and the pre-push hook still spell `GATE_RUN` — and nothing else; the + lint and test matrix stays in `ci.yml`. +- **`molmcp.evolution`** — `evaluate`, `EvaluationReport`, `EvaluationError`, + `EvalCase`, `Metrics`, `Challenger`, `ContractOutcome`, `ContractRunner`, + `ReplayFn`, `DEFAULT_SEEDS`, the seven reason constants (`ACCEPTED`, + `REGRESSION_FAILED`, `NO_PRACTICAL_GAIN`, `WORSE_*`) and the four `DROP_*` + thresholds. One implementation module, `evaluate.py`. +- **`molmcp.collection`** — `CollectionIndex`, `ContextPack`, `SearchHit`, + `SourceBinding`, `DEFAULT_CONTEXT_BUDGET` (16 000), `MAX_CONTEXT_BUDGET` + (32 000), `compose_context`, `open_ref`, `outline_source`, `packages_catalog`, + `search_scoped`. +- **`molmcp.discovery`** — `SCHEMA_VERSION`, `DiscoveryConfig`, `DiscoveryEngine`, + `DiscoveryQuery`, `CodeGraph`, `Node`, `Edge`, `Snapshot`, `CapabilityOverlay`, + `CatalogOverlay`, `OverlayContribution`, `load_overlays`, `load_catalog`, + `lint_graph`, … `schema.py` carries `SCHEMA_VERSION = 4`, `ANALYZER_VERSION = 2`. + `source/github.py` reaches the network only through `_transport()` returning a + `molmcp.components.git.GitTransport`, mapping `GitError` → `SourceError`. +- **`molmcp.middleware`** — `PathSafetyMiddleware`, `ResponseLimitMiddleware`, + `MissingAnnotationsError`, `ToolNamingError`, `assert_plane_tool_names`, + `validate_plane_tool_names`, `validate_tool_annotations`. +- **`molmcp.helpers`** — `run_safe`, `SubprocessResult`, `fence_untrusted`. +- **Providers** — `MolvisProvider` (`open`, `close`, `list_sessions`, `exec`, + `capabilities`, `refresh`, `poll_events`), `MolqProvider` (`list_jobs`, + `get_job`, `job_logs`, `list_destinations`, `list_queue`, `submit_job`, + `cancel_job`), `MolexpProvider` (workspace navigation, scaffold, and the + adoption tools). `providers/molexp/adopt/` is a 50-name stdlib core reaching + molexp through two injected seams. +- **Entry points** (`molmcp.providers`): `molexp`, `molq`, `molvis`. Console + script `molmcp = molmcp.cli:main`. Overlays use the parallel `molmcp.overlays` + group. ### Style summary -- **Naming** — modules are lowercase single words (`engine.py`, `resolve.py`, `graphstore.py`); private module-level helpers are `_leading_underscore` and are never re-exported. Every package `__init__.py` carries an explicit alphabetically-sorted `__all__`; `discovery/analyzers/`, `discovery/cache/`, `discovery/source/`, `discovery/store/`, `collection/`, `middleware/`, `helpers/`, `providers/molexp/adopt/` all re-export their members from one place. Provider subpackages re-export exactly one class (`MolqProvider`, `MolvisProvider`, `MolexpProvider`). -- **Tool declaration** — providers subclass `ProviderBase`, set `name` / `upstream` / `import_name` as `ClassVar`s, and decorate methods with `@tool()` from `providers/annotations.py`. `name=` is passed only where the Python name cannot be the wire name (`open_session` → `open`, `exec_code` → `exec`). `ProviderBase.register` binds methods (so `self` never reaches the tool schema) and raises on duplicate wire names. The `catalog` plane and `MolCraftsContextProvider` are the two places still using raw `@mcp.tool(annotations=…)` with a locally-defined `_READ_ONLY`. -- **Construction / injection seams** — no import-time construction of collaborators. Every plane's outbound dependency is a constructor-injected callable with a real default that imports the science package lazily: - - `MolqProvider(db_path=, allow_submit=, store_factory=, submitor_factory=, destinations_factory=)` — defaults `_molq_store` / `_molq_submitor` / `_molq_destinations`. - - `MolvisProvider(stage_factory=)` — default `_molvis_stage`; `probe()` is overridden so an injected factory makes the plane available with no `molvis` installed. - - `adopt.run_adoption(..., workspace_factory=molexp_workspace_factory, ingest=molexp_ingest)` — the same shape one layer down, keeping `adopt/` pure stdlib. - - `CollectionIndex(bindings, registry, metadata)` takes a duck-typed `registry` (`search`/`get`/`info`); molmcp ships no implementation, the seam only. - - `create_plane(collection=, provider=, config=)` lets tests inject instead of touching entry points. - - `DiscoveryEngine(DiscoveryConfig(...))` — tests always pass an explicit `cache_dir`. -- **Lazy optional science** — no module-level import of `molq` / `molvis` / `molexp` anywhere; every one is a function-body import. `ProviderBase.probe()` asks `importlib.util.find_spec` rather than importing; `require_upstream()` raises `RuntimeError` naming the distribution and the `pip install` line. Missing packages are a *silent omit* from catalogs (`discover_providers(only_available=True)`, `list_plane_infos`), and a loud failure only on explicit `molmcp serve `. -- **Dataclasses** — pervasive, `frozen=True, slots=True` for value objects on a wire or contract (`PlaneInfo`, `PlaneToggle`, `AppConfig`, `ServerConfig`, `SourceBinding`, `SearchHit`, `ContextPack`, `ToolSpec`, `Snapshot`, `Capability`, `LayoutLevel`, adoption plan types). Mutable-but-slotted where the object accumulates (`DiscoveryConfig`, `OverlayContribution`, `CodeGraph`). Enums are `StrEnum` so the JSON literal *is* the member value. -- **Error handling** — layer-owned exception types rather than bare `ValueError`: `ConfigurationError`, `SettingsError`, `SourceError`, `MissingAnnotationsError`, `ToolNamingError`, `SessionExistsError` / `SessionNotFoundError`, `AdoptionBlocked`, `HashMismatch` / `TransferError`, `LedgerMismatch`, `AnalyzerNotAvailable`. Discovery/plugin load paths swallow-and-log at `debug`/`warning` with a structured `failures` sink (`discover_providers`, `load_overlays`) so one broken plugin cannot take down startup; `create_plane` re-raises provider registration failures after `logger.exception`. Tool bodies return `{"ok": false, ...}` payloads with a `hint` naming tools **bare** rather than raising across the wire. -- **Config discipline** — no environment variables outside the two documented credential exemptions (`server.auth_token_env` read in `server._EnvironmentTokenVerifier`, `GITHUB_TOKEN` in `DiscoveryConfig.__post_init__`). Everything else is a settings key read through `load_settings` (`molq.database`, `molq.allowSubmit`, `molexp.workspace`, `knowledgeScope`, `cacheDir`). -- **Versioned contracts** — `SCHEMA_VERSION`/`ANALYZER_VERSION` in `discovery/schema.py`, `CONFIG_SCHEMA_VERSION` in `config.py`, `LEDGER_VERSION` in `adopt/ledger.py`, and `LEGACY_EXTRACT_DB_NAMES` for cache-file renames. +- **Root package** — `__all__` sorted; PEP 562 `__getattr__` over `_LAZY_EXPORTS`; + unknown names must raise `AttributeError`, which is load-bearing so + `from molmcp import cli` still resolves a submodule. +- **`cli`** — private `_` handlers, an `argparse` tree in `_build_parser`, + int exit codes, `ConfigurationError` surfaced as a message. +- **`server`** — public `create_*` plus `_`-prefixed helpers; free functions + returning `FastMCP`; keyword-only options; injected `collection` / `providers` / + `extras` seams; `ValueError` / `ConfigurationError` at build time. +- **`runtime`** — pure functions of `AppConfig`; overlays assembled exactly once + (entry-point overlays then `extras`); `OverlayLoadError`. +- **`provider` / `provider_sdk`** — `@tool(ANNOTATION, name=...)` sets a private + marker; `tool_specs()` walks the MRO; `register()` hands bound methods to + `mcp.tool`; `RuntimeError` from `require_upstream`, `ValueError` on a duplicate + wire name. +- **`host`** — frozen-slots `HostLayout` of home-relative tuples; `Path.home()` + resolved only at read time; bundle source passed in explicitly + (`resolve_bundle_source`, `None` ⇒ no-op); unknown host raises `ValueError` + *before* the no-op check. Stdlib only; must not import `client_config` / `cli` / + `server` / `providers` / `discovery`. +- **`components`** — `*Spec` / `*Catalog` / `*Error` families; frozen slots with + `__post_init__` validation; a single `os.replace` publishes a whole SHA + directory; typed error hierarchies, no silent fallbacks; reads no environment. +- **`provider_worker`** — NDJSON frame verbs; `spawn=` injected for tests; path + launch (`python -P child.py`), never `python -m`; teardown wraps the child + server's `_lifespan` with an idempotent `shutdown()` and one + `weakref.finalize` backstop; an isolation leak fails the child before `hello`. +- **`evolution`** — SCREAMING verdict/threshold constants; one `evaluate()` whose + `ContractRunner` / `ReplayFn` seams are keyword-only with no default, because a + default would have to be a real host; `EvaluationError(ValueError)`. Verdict + only — moves no pointer, reads no settings. +- **`collection`** — noun result types in `models.py`, verb page-builders in + `browse.py`; agent-facing failures are JSON `{"ok": false, ...}` payloads. +- **`discovery`** — phase modules over leaf packages; snapshot-keyed immutable + caches; per-file analyzer failures recorded rather than raised; versioned + contracts in `schema.py`. +- **Repo-wide** — Python ≥ 3.12, `src/` layout, `from __future__ import + annotations` everywhere, ruff (`E,F,I`, line length 88), Google-style + docstrings, and no environment variables outside the three exemptions in + `tests/test_no_env_switches.py`. ### Layer roles -Dependency rule from `## Architecture`: **dependencies point inward only.** Verified by reading every `from .` / `from molmcp` import in all 78 files — no violation found. In particular: nothing in `discovery/` imports `collection/`, `providers/`, `server`, or `config`; nothing in `providers/` imports `server`, `planes`, or `collection`; `collection/` does not import `discovery` (it goes through the duck-typed `SourceBinding` seam); no import cycle exists (`middleware/naming → annotations_validator` and `discovery/overlay/{catalog,conventions} → overlay/__init__` are one-directional). - | Module | Layer role | |---|---| -| `__main__.py`, `cli.py` | Layer 1 — entry point | -| `server.py` | Layer 1 — plane construction (`create_plane`); the only assembler | -| `__init__.py` | Layer 1 — package façade / re-export surface | -| `planes.py`, `client_config.py` | Layer 2 — multi-link on-demand catalog & client wiring | -| `middleware/*` | Layer 1/2 — server-build validation + request-time guards, owned by `server.py` | -| `mcp_provider.py` | Application — molcrafts knowledge plane's MCP surface | -| `collection/*` | Application — MCP-free retrieval/paging over sources | -| `runtime.py` | Application — `AppConfig` → engine + bindings composition root | -| `guide.py`, `source_scope.py` | Application — routing vocabulary and `knowledgeScope` policy (leaf; no molmcp imports except lazy `settings`) | -| `provider.py` | Layer 3 — provider contract (Protocol + entry-point discovery) | -| `providers/base.py`, `providers/annotations.py` | Layer 3 — shared provider infrastructure | -| `providers/molq/*`, `providers/molvis/*`, `providers/molexp/*` | Layer 3 — one plane each; MCP machinery in, science packages lazy | -| `providers/molexp/adopt/*` | Layer 3 (inner) — pure-stdlib adoption core behind two injected seams | -| `providers/molvis/session.py`, `capabilities.py`, `refresh.py`; `molexp/layout.py`, `scaffold.py` | Layer 3 (inner) — the MCP-free half of each provider | -| `helpers/*` | Layer 3 — utilities offered to downstream provider authors (no in-tree importer) | -| `discovery/engine.py` | Layer 4 top — orchestration | -| `discovery/extract.py`, `resolve.py`, `query.py` | Layer 4 middle | -| `discovery/ranking.py` | Layer 4 middle — retrieval scoring, consumed by `query.py` only | -| `discovery/lint.py` | Layer 4 — read-only graph consumer (does not bump `SCHEMA_VERSION`) | -| `discovery/store/*`, `source/*`, `cache/*` | Layer 4 bottom — persistence, snapshot resolution, caching | -| `discovery/analyzers/*` | Layer 4 bottom — per-language extraction, dispatched by extension | -| `discovery/overlay/*` | Layer 4 — post-resolution domain overlay seam (`molmcp.overlays` entry points) | - -**Shared value objects consumed by inner layers** (they sit under everything and depend on nothing in molmcp): - -- `discovery/schema.py` — the language-agnostic graph contract; the single most-imported module inside Layer 4, and the owner of `SCHEMA_VERSION` / `ANALYZER_VERSION`. -- `discovery/config.py` — `DiscoveryConfig` / `DEFAULT_EXCLUDES`, flowing inward from `runtime.py` and `cli.py` down to `source/` and `cache/`. -- `settings.py` — leaf; reached by `config.py` at module level and by Layer 3 providers via lazy in-function imports (inward, so compliant). -- `config.py` / `environment.py` — `AppConfig` is the outer-layer value object; `config.py` reaches `environment.py` lazily and `environment.py` reaches back only for `ConfigurationError`. -- `collection/models.py` — the collection layer's owned wire types, deliberately not `discovery.Node`, which is what keeps `collection/` independent of `discovery/`. -- `providers/annotations.py` — the shared `ToolAnnotations` vocabulary all Layer 3 planes depend on. +| `__init__.py`, `__main__.py`, `cli.py` | **L1 entry points**. `cli.py` is one of only two importers of `discovery` | +| `server.py` | **L2 composition root**. Sole consumer of the activated harness checkout | +| `runtime.py` | **L2 assembly**. The one place overlays are ordered and the one owner of `resolved_cache_dir`, kept so `server.py` need not import `discovery` | +| `planes.py`, `provider.py` | **L2** plane catalog and entry-point discovery | +| `provider_sdk.py` | **L2 public SDK** — authoritative for the L3 contract; `providers/base.py` and `providers/annotations.py` are compatibility shims onto it | +| `mcp_provider.py` | **L2** core knowledge adapter over `collection` | +| `client_config.py` | **L2** MCP JSON body; owns no paths | +| `host/` | **L2 host adapter** — the single host path table plus write primitives; stdlib-only, so `client_config` reads it without a cycle | +| `config.py`, `settings.py`, `environment.py`, `guide.py`, `source_scope.py` | **L2 application policy**, MCP-free | +| `gate.py` | **L2 CI-parity check** — reads the workflow and the pre-commit config as text and reports whether they still spell `GATE_RUN`. Runs nothing, spawns nothing, and is the only owner of that literal; `cli.py`'s `gate` handler adds no verdict of its own | +| `collection/` | **L2 retrieval façade** between `mcp_provider`/`cli` and `discovery` | +| `middleware/` | **L2 cross-cutting server policy** | +| `providers/`, `providers/molvis\|molq\|molexp` | **L3 provider planes** — import MCP machinery, science packages lazy-optional, bare tool names | +| `providers/molexp/adopt/` | **L3 inner core** — pure stdlib behind two injected seams | +| `provider_worker/` | **L3 out-of-process plane adapter** — `protocol.py` and `child.py` are stdlib-only leaves that must stay free of FastMCP | +| `discovery/` | **L4 MCP-free discovery engine**, itself `engine → extract/resolve/query → store/source/cache`, with `schema.py` as the language-agnostic contract | +| `components/`, `helpers/`, `evolution/` | **shared stdlib leaves, not a layer** — imported by both outer (`server`, `cli`) and inner (`discovery/source/github.py`) modules; none re-exported from `molmcp` | +| `skill/` | **packaged data**, installed only by `molmcp init` | +| `introspection/`, `registry/`, `providers/lammps\|molpack\|molpy\|molrs` | **no role** — stale `__pycache__` residue only | diff --git a/.claude/notes/harness-contract.md b/.claude/notes/harness-contract.md new file mode 100644 index 0000000..febd2e5 --- /dev/null +++ b/.claude/notes/harness-contract.md @@ -0,0 +1,34 @@ +# Harness contract — two long-lived rules + +Two rules only. Everything else about the harness — the catalog keys, the +`official` / `gate` / `canary` labels, the licence table, the example file — +lives on `docs/concepts/harness.md`, next to the example that demonstrates it. +Restating any of it here would create a second copy, and the copy is the one +that goes stale. + +## 1. Two repositories, not one rename + +`MolCrafts/harness` is a **new empty repository**. It is not +`MolCrafts/molcrafts-harness` renamed. + +`MolCrafts/molcrafts-harness` was the plugin marketplace. It is archived or +deleted **only after cutover**, never before, and that step needs its own +authorisation. Until then it keeps its own history and its own MIT licence. + +Why a rename was refused: it would carry the old marketplace layout and every +stale install instruction into the new repository's first commit; it would +leave a GitHub redirect, so a host still configured against the old address +would keep working and nobody would learn they were on it; and it would carry +the old licence across as a default, making a licensing decision by accident. + +The new repository holds agent tooling only. Provider repositories (molq, +molexp, molvis, molpy) do not move into it. + +## 2. Identity is a Git SHA + +A harness commit is identified by its 40-character lowercase Git SHA and by +nothing else — no version number, no `latest`, no tag, no branch. + +The SHA is not written into the catalog file; the caller that unpacked the tree +passes it to `load_harness_catalog`. A file stating its own SHA could disagree +with the tree it sits in, and nothing would be able to say which was wrong. diff --git a/.claude/notes/notes.md b/.claude/notes/notes.md index 1f33e57..7c4592e 100644 --- a/.claude/notes/notes.md +++ b/.claude/notes/notes.md @@ -2,6 +2,136 @@ Evolving architectural decisions. Appended by `/mol:note`; newest first. + +## 2026-09-07 — spec 引用别的模块时,必须当场核实再写进 Design + +autonomous-harness-evolution 那条 16 员链上,**四条 spec 的 Design 引用了并不存在 +的东西**,全部在实现阶段才炸: + +- spec 05:「FastMCP 4 没有 `mcp.lifespan` 属性」——它有;「`_lifespan` 可能为 + `None`」——永不为 None;「dict 返回值需要 return 注解才有 structured content」 + ——不需要。 +- spec 07:`resolve_bundle_source` 被定为「唯一解释入口」,却没有任何调用方, + `--source /不存在` 会静默降级。 +- spec 08:把 `AppConfig.cache_dir` 当成总有值——它默认 `None`,导致没配 + `cacheDir` 的用户 harness 开箱即坏。 +- spec 12:`ActivationUnboundError` 仓里根本没有;而且 04 把未绑定状态做成了 + 不可构造(`Activation()` 直接 `TypeError("use Activation.bind")`)。 +- spec 13:`from molmcp.evaluate import evaluate`,签名 `Path -> bool`——真实符号 + 在 `molmcp.evolution.evaluate`,签名是 8 参数返回 `EvaluationReport`。 + +共同点:这些 spec 是**一次性批量起草**的,谁都没去跑一下。 + +**Rule**: spec 起草时凡引用另一个模块的类型名、字段、异常或签名,先 +`uv run python -c "import ...; print(inspect.signature(...))"` 核一遍,再写进 +Design。跨 spec 链尤其如此——后一条引用前一条**交付的**符号,不是前一条 spec +里**写的**符号。 + + +## 2026-09-07 — golden 必须是独立字面量,且必须真跑反例 + +本链两个回归带着**恒真断言**落库,同一个模式:一个常量既喂给被测函数当输入、 +又当断言的期望值,改它两边一起动,断言永远通不掉。两次都是**执行反例控制** +时才暴露,code review 看不出来。 + +**Rule**: 测试与示例里的 golden 与构造输入的字面量**分开各写各的**;每个 golden +至少跑一次「改坏 → 必须失败 → 还原」。控制项自己也要验:如果一个控制"通过"了, +那说明该 golden 是空的,先修 golden 再说。 + + +## 2026-09-07 — 往包门面加符号之前,先 grep 现有 `__all__` + +同一批 spec 里撞了三次: + +- spec 10 与 spec 11 都要从 `molmcp.evolution` 导出名为 `Candidate` 的东西—— + 一个是「被提议的补丁」,一个是「待评估的检出」。改名 `Challenger` 才解开。 +- spec 07 声明 `HOSTS: dict[Host, HostLayout]` 于 `host/layout.py`,spec 15 声明 + `HOSTS: tuple[Host, ...]` 于 `host/install.py`。 +- spec 09 与 spec 10 对**同一个包**指定了不同的测试目录(10 还显式排除了 09 选的)。 + +**Rule**: 起 spec 时若要往某个包的 `__all__` 加符号,先读那个 `__all__`,再读 +同链其它 spec 的 Files 段。撞名不是实现细节,是两个概念抢一个词,必须在 spec +阶段解决。 + + +## 2026-09-07 — 测试目录用 `test_` 前缀镜像 `src/` + +仓里两种约定都有先例(`tests/discovery/` `tests/collection/` 无前缀; +`tests/test_components/` `tests/test_provider/` 有前缀),于是 spec 09 与 10 各 +选一种、互相矛盾。已统一。 + +**Rule**: `src/foo/bar.py` 的单测放 `tests/test_foo/test_bar.py`。一个源码包的 +测试只放一个目录,不得分散。 + + +## 2026-09-07 — 「不得依赖 X」用 AST 查 import,不要全文 grep 子串 + +`test_wiki.py` 曾禁止 `wiki.py` 全文出现小写 `github`,结果实现被迫写成 +`_FORGE_SCHEME = "git" + "hub:"` ——而那段代码的作用恰恰是**拒绝** forge URL, +是隔离的证据而不是违反。测试自己也得靠 `"create_" + "stack"` 躲开自己的扫描。 +钝 grep 同时过宽(命中 docstring 与拒绝逻辑)又过窄(躲不过字符串拼接)。 + +**Rule**: 依赖隔离断言走 AST——遍历 `Import` / `ImportFrom`,把相对 import 解析 +成绝对点分路径再比。只有 `importlib.import_module("...")` 这种 AST 看不见的 +动态导入才补一条针对**点分模块路径**的文本检查。 + + +## 2026-09-07 — FastMCP 4.0.0b5 lifespan 事实（推翻 spec 05 的三条前提） + +Spec 05 的 Design 写了三条关于 FastMCP 4 的断言，实测**全错**。它们已在 +`provider_worker/` 的 docstring 里改正，但 06–16 的 spec 正文可能仍带着旧 +说法——照抄前先核对源码。 + +1. **`mcp.lifespan` 存在**，是继承来的 `AggregateProvider.lifespan` + (`fastmcp/server/providers/aggregate.py:345`)：无参 `@asynccontextmanager`， + 聚合的是**被 mount 的 provider** 的 lifespan。它与构造函数 `lifespan=` + 存进 `_lifespan` 的那个 callable 是**两个不同对象、不同签名**。 + 设计上刻意不用它——但不能说它「不存在」。 +2. **`FastMCP._lifespan` 永不为 `None`**：`__init__` 在未传 `lifespan=` 时 + 回落到 `default_lifespan`（`server/server.py:404-408`）。任何 + `if previous is None` 分支都是防御性死代码，写注释说明，别当正常路径。 +3. **dict 返回值无需 return 注解**即可产出 structured content + （实测 `ToolResult.structured_content == {"text": "ping"}` 两种情形一致）。 + 所以 worker 线协议**不带** return 字段——别为它加。 + +**Rule**: 引用 FastMCP 私有属性前，先读 +`.venv/.../fastmcp/server/` 的对应源码核实；`_lifespan_manager` +(`server/mixins/lifespan.py:169`) 做的是 +`enter_async_context(self._lifespan(self))` 并把 yield 值缓存成 +`_lifespan_result`——**任何包装 `_lifespan` 的代码必须把前一个 lifespan 的 +yield 值透传出去**，吞掉它就等于悄悄拿走了服务器的应用状态。 + + +## 2026-09-07 — worker 子进程的 FastMCP 隔离边界在 provider.py + +`provider_sdk.py` 早就把 `FastMCP` 放在 `TYPE_CHECKING` 下，但它 +`from .provider import Provider`，而 `provider.py` 当时是**模块级** +`from fastmcp import FastMCP`——于是 `import molmcp.provider_sdk` 照样把整个 +FastMCP 栈拖进任何进程。spec 05 的 child 必须 import `ProviderBase`， +AC-002（子进程无 fastmcp）因此不可能成立。已把那一行移到 `TYPE_CHECKING` +下（行为不变：该文件有 `from __future__ import annotations`，`FastMCP` 只出现在 +docstring 与被字符串化的 `register` 注解里）。 + +**Rule**: `molmcp.provider_sdk` 及其依赖链（`provider.py`）是**子进程可安全 +import 的边界**——不得在这条链上新增模块级 FastMCP / `molmcp.server` import。 +`molmcp/__init__.py` 与 `provider_worker/__init__.py` 是 PEP 562 惰性门面， +`__getattr__` **必须**对未知名抛 `AttributeError`：CPython 的 +`_handle_fromlist` 靠它回落到子模块导入，`from molmcp import cli/settings/ +runtime/client_config` 等十余处调用点依赖这一点。 + + +## 2026-09-07 — ruff 的 first-party 判定随文件存在与否翻转 + +ruff 的 isort 按**目标模块文件是否存在于 `src/` 下**判 first-party。于是 +RED 阶段写的 import 块（模块尚不存在 → 判 third-party）会在 GREEN 之后变成 +I001。更糟的是 `.ruff_cache` 会掩盖它：751e874 就这样带着 +`tests/test_components/test_models.py` 的 I001 落库，本地暖缓存全绿而**干净 +检出必然挂 CI lint**（已修，见 fb4c348）。 + +**Rule**: 提交前用 `rm -rf .ruff_cache && uv run ruff check src tests` 复核 +——CI 与新克隆跑的都是冷缓存。TDD 写测试时，先造出目标模块的空壳或事后 +`ruff check --fix`，别相信 RED 阶段的 lint 结果。 + ## 2026-08-02 — molvis provider = 工作台原语,不是接口翻译层 molmcp 对 molvis 的角色定位:**把「活着的 Python 会话」借给 agent,而不是替 @@ -93,3 +223,122 @@ Same placement rule as `providers/molexp/`. Contract: 突变测试）中允许；裸 `# type: ignore` 不允许。** - **MCP payload 契约测试钉序列化字面量**（如 `"resolved"`），不引用枚举成员—— 测的是 wire format。 + + +## [2026-09-08] 缝把函数假掉时，至少要一条测试驱动真函数 + +`harness-evo-01-sources` 期间，`Settings.harness` 从 dict 改成 tuple 后 +`server._harness_locator()` 对**每一个**安装都抛 `AttributeError`，`molmcp serve` +已断——而全量套件 1852 条全绿。原因：`tests/test_stack.py` 通过 `_wire` 缝注入 +一个假的 locator，`grep -rn "_harness_locator" tests/` 唯一的命中是一个**测试 +名字**，没有任何测试调用过真函数。缝越好用，越没人调用真货。 + +**Rule**：为某个函数造了测试缝之后，必须同时留至少一条不走缝、直接调用真函数的 +测试。缝证明的是调用方编排正确，不是被缝掉的那个函数还能跑。 + + +## [2026-09-08] 翻转 `_SCHEMA` 类型会静默解锁旧类型正在拒绝的写路径 + +`settings._SCHEMA["harness"]` 从 `dict` 改成 `list` 的瞬间，两条 CLI 写路径失去 +保护：`_parse` 的 `expected is dict` 分支（抛 "set a member instead"）不再命中， +改走 `expected is list` 返回 `[value]`；`add_value` 的 `_SCHEMA.get(top) is not +list` 守卫不再触发，直接 append 裸字符串。两者都在 `write_settings_file` 之前 +**无任何校验**。而 `_reject_unknown` 位于 `load_settings` 之下，于是下一条命令起 +`config list/get/set/add/remove` 与 `serve` 全部 exit 2，**没有任何 CLI 能救回**， +只能手改 JSON。 + +**Rule**：改 `_SCHEMA` 里某个键的类型时，先列出 `_parse` / `set_value` / +`add_value` / `remove_value` / `_resolve` 中按**旧类型**分支的每一处，逐条确认新 +类型下谁还在拒绝、谁开始放行。类型不只是校验规则，它同时是这些动词的调度键。 +配套：元素是对象的 list 用 `_OBJECT_LISTS` 声明，两个字符串动词读表拒绝， +不在函数体里写死键名。 + + +## [2026-09-09] harness 列表跨层是「整份替换」，并集已放弃 + +链 01 把「跨层取并集」记为欠链 03 的债，但链 01 同时发布了钉住相反行为的东西： +`tests/test_settings.py` 的 `test_harness_is_a_list_setting_with_no_merge_channel` +与 `test_the_most_specific_layer_replaces_the_list_rather_than_merging`、 +`settings.py` 里 `harness` 不属于任何合并通道、以及 `docs/concepts/harness.md` +的相应段落——最后一条还是构建强制的（`test_harness_catalog_fixture.py` 会解析 +该页 JSON 并逐条构造真的 `HarnessSource`）。 + +链 03 的决定：**并集放弃，不是再往后推**。最具体的层整份胜出是自洽规则，没有任何 +东西需要打破它；而 `harness` 不入任何合并通道，正是这条规则不用写代码就成立的原因 +（`load_settings` 的默认分支「最后一次赋值胜出」+ `settings_layers` 低优先级在前）。 + +注意与 `_MERGED_LISTS` 成员方向相反：`excludes` / `knowledgeScope` 等用 `extend` +低→高累积，所以**用户文件**的条目活下来；`harness` 是**local 文件**的列表整份取代 +用户文件的。两个 list 设置相隔十几行、方向相反，`ac-006` 在同一个测试里同时断言两者 +就是为了让这件事写在测试里而不是留给人在安装时踩。 + +**Rule**：想让 harness 跨层合并之前，先改上面那两条测试和那页构建强制的文档； +它们是这个决定的落点，不是随手可绕的断言。 + + +## [2026-09-09] harness store 刻意不用 git 支撑，理由和触发阈值 + +问过一次：harness 仓本来就是 clone 下来的，`ImmutableGitStore` 为什么还要把每个 +commit 解压成一份完整的树？实测过开销：demo 仓每个 commit 644K，真实 harness 仓 +`.git` 12M / 工作区 13M，所以激活 N 个版本约等于 N 份工作区，而 git 用共享对象只需 +一份历史。空间上 git 明显更省，`git worktree` 还能让多个 commit 同时物化（盲测 A/B +正需要两棵树并存），回滚也能到任意 commit 而不只是 `previous`。 + +**仍然不做，三个代价换不回来：** + +1. **不可变性会丢。** 现在的树解压一次后永不改动——这是 `ImmutableGitStore` 里 + "Immutable" 的实质，服务中的 harness 不能被就地篡改。worktree 是活的检出。 +2. **远程路径会多一个 git 依赖。** 今天远程是纯 HTTP + tarfile，没有 git 二进制也能 + 跑；改成 clone 就必须有，且要处理 `--depth` / partial clone。 +3. **落盘契约要变。** `/harness/commits//tree` 与 `metadata.json` 的 + provenance + `ShaConflictError` 是 CLAUDE.md 列为「不可随意变更」的那类，需要 + bump、旧目录处理、迁移路径。 + +还有一个不显然的坑：直接在用户的工作检出里 `git worktree add`，会把 molmcp 的 +worktree 写进**用户仓库**的 `.git/worktrees`。干净做法是 molmcp 在缓存里维护自己的 +裸镜像（本地源可 `git clone --local` 硬链对象），再从镜像开 worktree——但那是又一个 +要维护的东西。 + +**Rule**：在有人真的激活到几十个版本、或者 `molmcp cache` 清理不足以应付之前，不要 +重开这个话题。真要做，先写 spec：上面第 1 条是真会丢的性质，必须先说清用什么补 +（worktree 建完 `chmod -R a-w`？还是接受可改并说明为什么可以），而不是默认它无所谓。 + + +## [2026-09-10] 盲测评估器：机制随 harness 走，用例归项目 + +`/mol:evo`（skill）+ `harness-actor` + `harness-observer` 是 **harness 组件**，作为 +`evo` bundle 随 harness 安装；`scripts/harness_cases.py` 与 `scripts/harness_eval.py` +留在 molmcp。 + +分界线是 `harness_cases.py` 自己写下的那句：「Every case tests a rule `CLAUDE.md` +already states」。用例编码的是**某个仓库**的规则——molmcp 的用例拿到 molpy 上就是 +胡话。而盲测协议（manifest 先写、actor 只读且不见判据、observer 只见标签、只有 +Python 门能判胜负）在哪个仓库都一样。 + +所以 skill 不许硬编码 molmcp 的路径：它声明自己需要什么（带 `id` / `graduated` / +`task` / `expect` / `forbid` 的用例集，加一个把 manifest + observation 变成裁决的 +命令），把 molmcp 那两个文件只当**示例**写。项目两样都没有 → 停下来说清楚。 + +**Rule**：往 evaluator 里加东西前先问它是协议还是判据。协议进 harness 仓，判据留 +项目仓。skill 里出现第二个写死的 molmcp 路径，就是这条被违反了。自己编用例来填空 +等于什么都没测量。 + +**另见**：冠军/挑战者不是「两个激活的 commit」——激活指针每源只有一个 `active`。 +成对的是 `previous`（冠军）与 `active`（挑战者），靠 store 的发布不可变且只增， +两棵树才能并存被读。`worse_tokens` / `worse_latency` 这条路走不到：两侧都钉死为 0。 + + +## [2026-09-10] `.claude/agents/harness-*.md` 是临时副本,等 PR 落地后删 + +`harness-actor` / `harness-observer` 的正主已经是 +`Roy-Kid/molcrafts-harness`(PR #1,`plugins/mol/agents/`)。molmcp 树里这两份是 +同内容副本,只为在 PR 合并前不分叉。 + +**Rule**:PR #1 合并后,删掉 molmcp 的 `.claude/agents/harness-actor.md` 与 +`harness-observer.md`,把 `tests/test_harness_agents.py:26-29` 从 `REPO/.claude/agents` +改成读已安装位置(或改成对 harness 仓的契约测试)。在那之前改这两个文件, +**两边都要改**——只改一边就是本条被违反。 + +`scripts/harness_cases.py` / `scripts/harness_eval.py` / `src/molmcp/evolution/` +不搬:那是判据和裁决门,归项目。见 +[[evaluator-splits-harness-from-project]]。 diff --git a/.claude/notes/open-questions.md b/.claude/notes/open-questions.md index a22f4bb..090dd93 100644 --- a/.claude/notes/open-questions.md +++ b/.claude/notes/open-questions.md @@ -4,3 +4,19 @@ Intentional, or should `check` grow a type-check step? - No coverage tooling (`pytest-cov` absent), so `mol_project.build.coverage` is unset. Add if coverage gating is wanted. + +- **Harness overlay 会给 discovery 缓存分区，且没有保留策略。**（2026-09-07，spec 08） + checkout 的 capability overlay 进入 `DiscoveryEngine(overlays=)` 后会改变 + `_build_identity`，于是**每个 harness SHA** 得到自己的 + `cache/profiles//graph.db` 与 `extract_cache.analyzer_version`。 + 换一次 harness ref 就多一棵图缓存树，没有任何东西回收旧的。CLAUDE.md 的 + 「stranded multi-gigabyte orphan」正是在讲这个。spec 08 明确不解决。 + 待定：按 SHA 数量还是按时间剪除？`molmcp cache` 子命令要不要看得见 harness 分区？ + +**Evidence added 2026-09-08.** `uv run ty check src/molmcp/settings.py` failed on a +diagnostic dating to `1fad8f6` (`HarnessSource(**entry)` — "Argument expression after +`**` must be a mapping with `str` key type"). `mol_project.build.check`, +`.pre-commit-config.yaml` and CI all passed around it silently for two links. The +diagnostic is now fixed, but nothing would have caught the next one. Wiring `ty` into +`check` is a CI-parity change: `.pre-commit-config.yaml` mirrors `.github/workflows/ci.yml` +step-for-step and both move in one commit. diff --git a/.github/workflows/official-gate.yml b/.github/workflows/official-gate.yml new file mode 100644 index 0000000..618958d --- /dev/null +++ b/.github/workflows/official-gate.yml @@ -0,0 +1,58 @@ +# The repository's single required GitHub check. +# +# GitHub matches a required check on a job's `name:`, so `official/gate` on the +# pull-request job below is the one line that makes the check exist. What it +# runs is `uv run molmcp gate` — the same sentence `.pre-commit-config.yaml` +# hands the `official-gate` hook as its `entry:`, character for character. +# `uv sync --extra dev` is the prior Install step and is never folded into that +# token, and no `run:` here holds a `${{ }}` expression: what an expression +# expands to on a runner is not what parity compared. The `${{ }}` in +# `concurrency:` is deliberate — expressions are legal everywhere but a `run:`. +# +# Lint and tests are not here. They stay on ci.yml's OS/Python matrix; running +# them again under this name would report one verdict twice, more slowly. +name: Official Gate + +on: + pull_request: + branches: [master, dev] + schedule: + # Monday 06:00 UTC. The timer is not a knob: it re-checks that the wiring + # is still intact during a week with no pull request. + - cron: "0 6 * * 1" + workflow_dispatch: + +concurrency: + group: official-gate-${{ github.ref }} + cancel-in-progress: true + +jobs: + official-gate: + name: official/gate + if: github.event_name != 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate + + official-gate-schedule: + name: official/gate (schedule) + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 3561cd8..540e35f 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -1,5 +1,9 @@ # Managed by prek: `prek install` -# CI parity: same shell commands as .github/workflows/ci.yml (no tox wrapper). +# CI parity, pair 1: ci-lint / ci-test are the same shell commands as +# .github/workflows/ci.yml's Lint / Test steps (no tox wrapper). +# CI parity, pair 2: official-gate's entry: is the same literal as the +# official-gate job's run: in .github/workflows/official-gate.yml — bare, so +# that the two are equal character for character. # tox remains in [project.optional-dependencies] dev for optional local isolation. default_install_hook_types: [pre-commit, pre-push] @@ -36,3 +40,15 @@ repos: pass_filenames: false always_run: true stages: [pre-push] + + # The bare literal, on purpose: not `entry: uv` plus `args:`, and not + # the `bash -c 'uv sync && …'` wrapper the two hooks above carry. It has + # to equal the official-gate job's `run:` as a string, and a wrapper is + # a different string. Push tier only — the commit tier stays fast. + - id: official-gate + name: "official/gate (same as official-gate.yml)" + entry: uv run molmcp gate + language: system + pass_filenames: false + always_run: true + stages: [pre-push] diff --git a/AGENTS.md b/AGENTS.md index 2159b2e..ea12ff2 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -30,13 +30,14 @@ mol_project: ## What this repo is -molmcp is multi-plane MCP for MolCrafts: **one product domain per MCP -connection** (`molmcp serve `). Planes include `catalog`, -`molcrafts` (knowledge/discovery), `molvis`, `molq`, `molexp`. Science -APIs are discovered via the knowledge plane and never mirrored as MCP -tools. Pure Python (>= 3.12), `src/` layout, managed with uv. - -**Protocol:** MCP **2026-07-28** via FastMCP **4.0.0b1** + MCP Python SDK +molmcp is FastMCP-composed MCP for MolCrafts: **`molmcp serve`** starts the +molcrafts core (knowledge plus `list_planes` / `route`) and mounts enabled +providers with official namespaces (`molvis_open`). `molmcp init ` +installs managed skills (`molcrafts`, `molexp-plan`) and one MCP entry. Science APIs are discovered +via the core and never mirrored as MCP tools. Pure Python (>= 3.12), +`src/` layout, managed with uv. + +**Protocol:** MCP **2026-07-28** via FastMCP **4.0.0b5** + MCP Python SDK v2 (`mcp>=2`). Do not pin FastMCP back to 3.x without an explicit decision. ## Where things live @@ -81,10 +82,10 @@ For non-trivial work, prefer: Layered; dependencies point inward only: -1. `cli.py` / `__main__.py` → `server.create_plane(plane)` → one plane - only (`planes.py` catalog + molcrafts knowledge + one provider). -2. Multi-link on-demand: clients connect separate MCP servers - (`catalog`, `molcrafts`, `molvis`, …). No mega-mount. +1. `cli.py` / `__main__.py` → `create_stack()` (default `molmcp serve`) + or `create_plane(plane)` for a focused debug process. +2. FastMCP composition: molcrafts core mounts providers with namespaces + (`molvis_open`). `molmcp init --disable` omits a mount. 3. Providers (`providers/`) import MCP machinery; science packages stay lazy optional. Bare tool names; server name is the plane id. 4. `discovery/` is itself layered: @@ -99,6 +100,25 @@ Layered; dependencies point inward only: +## CI parity: two pairs, both in one commit + +`.pre-commit-config.yaml` is the local half of two workflows, and each pair is +one string copied into two files. Change one side without the other and the +copies drift, so change both in the same commit. + +1. **Package matrix.** The `ci-lint` / `ci-test` hooks run the same shell + commands as the Lint / Test `run:` steps of `.github/workflows/ci.yml`. That + workflow stays the OS/Python matrix, and `mol_project.ci.config` keeps + pointing at it. +2. **Required check.** The `official-gate` hook's `entry:` is the same literal + as the `run:` of the `official-gate` pull-request job in + `.github/workflows/official-gate.yml` — both `uv run molmcp gate`, bare. + `uv sync --extra dev` is a prior Install step, not part of the compared + token, and no wrapper goes around either side. `src/molmcp/gate.py` owns + that literal (`GATE_RUN`) and the two files are its serialized copies; + `molmcp gate` is what checks they still agree. The GitHub required check is + named `official/gate`, which is that job's `name:`, not its id. + ## Discovery ranking & the call graph Capability discovery is a **retrieval** problem, not graph navigation. The diff --git a/CLAUDE.md b/CLAUDE.md index 987a86f..0795838 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -30,13 +30,15 @@ mol_project: ## What this repo is -molmcp is multi-plane MCP for MolCrafts: **one product domain per MCP -connection** (`molmcp serve `). Planes include `catalog`, -`molcrafts` (knowledge/discovery), `molvis`, `molq`, `molexp`. Science -APIs are discovered via the knowledge plane and never mirrored as MCP -tools. Pure Python (>= 3.12), `src/` layout, managed with uv. - -**Protocol:** MCP **2026-07-28** via FastMCP **4.0.0b1** + MCP Python SDK +molmcp is FastMCP-composed MCP for MolCrafts: **`molmcp serve`** starts the +molcrafts core (knowledge plus `list_planes` / `route`) and mounts enabled +providers with official namespaces (`molvis_open`). `molmcp init ` +installs managed skills (`molcrafts`, `molexp-plan`) and one MCP entry. +Science APIs are discovered via the core and never mirrored as MCP tools. +Pure Python (>= 3.12), +`src/` layout, managed with uv. + +**Protocol:** MCP **2026-07-28** via FastMCP **4.0.0b5** + MCP Python SDK v2 (`mcp>=2`). Do not pin FastMCP back to 3.x without an explicit decision. ## Where things live @@ -96,10 +98,10 @@ says so, and `./molcrafts.json` is not auto-loaded (`--config` still works). Layered; dependencies point inward only: -1. `cli.py` / `__main__.py` → `server.create_plane(plane)` → one plane - only (`planes.py` catalog + molcrafts knowledge + one provider). -2. Multi-link on-demand: clients connect separate MCP servers - (`catalog`, `molcrafts`, `molvis`, …). No mega-mount. +1. `cli.py` / `__main__.py` → `create_stack()` (default `molmcp serve`) + or `create_plane(plane)` for a focused debug process. +2. FastMCP composition: molcrafts core mounts providers with namespaces + (`molvis_open`). `molmcp init --disable` omits a mount. 3. Providers (`providers/`) import MCP machinery; science packages stay lazy optional. Bare tool names; server name is the plane id. 4. `discovery/` is itself layered: @@ -110,6 +112,25 @@ Layered; dependencies point inward only: +## CI parity: two pairs, both in one commit + +`.pre-commit-config.yaml` is the local half of two workflows, and each pair is +one string copied into two files. Change one side without the other and the +copies drift, so change both in the same commit. + +1. **Package matrix.** The `ci-lint` / `ci-test` hooks run the same shell + commands as the Lint / Test `run:` steps of `.github/workflows/ci.yml`. That + workflow stays the OS/Python matrix, and `mol_project.ci.config` keeps + pointing at it. +2. **Required check.** The `official-gate` hook's `entry:` is the same literal + as the `run:` of the `official-gate` pull-request job in + `.github/workflows/official-gate.yml` — both `uv run molmcp gate`, bare. + `uv sync --extra dev` is a prior Install step, not part of the compared + token, and no wrapper goes around either side. `src/molmcp/gate.py` owns + that literal (`GATE_RUN`) and the two files are its serialized copies; + `molmcp gate` is what checks they still agree. The GitHub required check is + named `official/gate`, which is that job's `name:`, not its id. + ## First-party providers - Path: `src/molmcp/providers//` + `molmcp.providers` entry point. diff --git a/README.md b/README.md index 6d89c7b..e738aa8 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ Multi-plane MCP for the MolCrafts ecosystem. -**Protocol:** MCP **2026-07-28** via **FastMCP 4.0.0b1** (+ MCP Python SDK v2). +**Protocol:** MCP **2026-07-28** via **FastMCP 4.0.0b5** (+ MCP Python SDK v2). Handshake-era clients still work — FastMCP 4 negotiates per connection. Optional science packages (`molvis`, `molq`, `molexp`, …): if not installed, @@ -10,20 +10,20 @@ that plane is **omitted from catalogs and client configs** (silent). Explicit `molmcp serve ` still errors with an install hint. This is runtime behavior — not a test skip. -**One product domain per MCP connection** (separate process / server name). -There is no mega-server under `molmcp`. **Client default: all planes on.** -Turn planes off with `--disable` (and back on with `--enable`). +**`molmcp serve`** (no plane) starts the **molcrafts core** and FastMCP-mounts +enabled providers into that one process (`molvis_open`, `molq_list_jobs`, …). +**`molmcp init `** writes that one MCP entry and the managed skills +(`molcrafts`, `molexp-plan`). +`--disable molcrafts` errors; `--disable molq` omits that mount. -| Plane | Command | Role | -|-------|---------|------| -| `catalog` | `molmcp serve catalog` | Bootstrap: `list_planes`, `route(task)` | -| `molcrafts` | `molmcp serve molcrafts` | Knowledge pages (packages → outline → open) | -| `molvis` | `molmcp serve molvis` | Live viewer session (`open` / `exec` / `poll_events`) | -| `molq` | `molmcp serve molq` | Job store + opt-in submit/cancel | -| `molexp` | `molmcp serve molexp` | Workspace layout + scaffold + data-directory adoption | +| Command | Role | +|---------|------| +| `molmcp serve` | Composed core + provider mounts | +| `molmcp serve molvis` | Debug: vis-only process, bare `open` | +| `molmcp init grok` | User-level skills + MCP JSON | -Science APIs are **never** MCP tools. Discover them on the `molcrafts` plane, -then call them from agent Python or inside `molvis` `exec`. +Science APIs are **never** MCP tools. Discover them on molcrafts (`packages` → +`open`), then call them from agent Python or `molvis_exec`. ## Client config (default: everything) @@ -31,16 +31,15 @@ One standard `mcpServers` JSON, which every host reads — Claude Code and Cursor natively, Grok alongside its own `config.toml`. ```bash -molmcp client # all planes, to stdout -molmcp client --disable molq --disable molexp -molmcp client --disable molq --enable molq # re-enable after a disable -molmcp client claude -o ~/.claude.json +molmcp init grok # skills + composed serve +molmcp init grok --disable molq --disable molexp +molmcp init grok --disable molq --enable molq # re-enable after a disable +molmcp init claude ``` -An optional host (`claude`, `cursor`, `grok`) picks the default output path; -the JSON itself is identical for all of them. A disabled plane is simply -absent from the map. Tool ids look like `molvis__open`, not -`molmcp__molvis_open`. +Host is required (`grok`, `claude`, `cursor`, `codex`). JSON is one +`molcrafts` entry running `molmcp serve`, with `--disable` flags for omitted +mounts. Tool ids look like `molcrafts__molvis_open`. > In Grok, `~/.grok/config.toml` outranks the JSON sources. If an old molmcp > entry lives there it still wins — `grok inspect` shows each server's origin. @@ -86,11 +85,11 @@ to be started next to. ```bash uv run molmcp planes # list planes -uv run molmcp client # client config, all planes on +uv run molmcp init grok # skills + MCP config uv run molmcp config list # resolved settings uv run molmcp route "draw dopamine" -uv run molmcp serve catalog # one plane per process -uv run molmcp serve molvis +uv run molmcp serve # composed core + mounts +uv run molmcp serve molvis # debug one plane uv run molmcp search "Conformer" # offline index search uv run molmcp index uv run molmcp cache # index size; --prune / --gc / --vacuum to reclaim @@ -105,12 +104,11 @@ uv run pytest -v ## Design rules -1. **Multi-link on-demand** — one process = one plane = one MCP server name. -2. **Bare tool names** — the plane id is the server name, so a tool registers - as `open` and the client shows `molvis__open`. +1. **FastMCP composition** — `molmcp serve` is molcrafts + namespaced mounts. +2. **Bare register, namespaced mount** — a provider registers `open`; the stack + exposes `molvis_open`. Debug `molmcp serve molvis` still shows `molvis__open`. 3. **No science tool mirror** — no `show_smiles` / `draw_dopamine`; discovery + Python. -4. **Providers** register via `molmcp.providers` entry points and are served with - `molmcp serve `. +4. **Providers** register via `molmcp.providers` entry points. 5. **No environment switches** — configuration is settings and CLI flags, so `molmcp config list` is the whole truth. diff --git a/docs/concepts/architecture.md b/docs/concepts/architecture.md index 6667d4c..1ee41c4 100644 --- a/docs/concepts/architecture.md +++ b/docs/concepts/architecture.md @@ -1,7 +1,10 @@ # Architecture -molmcp is multi-plane MCP infrastructure for MolCrafts. **Each MCP connection -serves exactly one plane** (product domain). Clients link planes on demand. +molmcp is FastMCP-composed MCP infrastructure for MolCrafts. **`molmcp serve`** +starts the **molcrafts core** (knowledge pages plus `list_planes` / `route`) +and **mounts** enabled providers into that process with official namespaces +(`molvis_open`). `molmcp init --disable molq` omits a mount. Providers +can still be served alone for debugging (`molmcp serve molvis`). **Protocol alignment:** servers run on **FastMCP 4 / MCP SDK v2**, which speak MCP **2026-07-28** (sessionless `server/discover`) while still serving older @@ -10,16 +13,20 @@ handshake-era clients. ``` MCP clients (Claude, Grok, …) │ - │ separate stdio (or HTTP) links — connect only what you need + │ one stdio: molmcp serve │ - ├── catalog list_planes / route - ├── molcrafts packages / outline / open / search / compose - ├── molvis open / exec / poll_events / … - ├── molq list_jobs / submit_job / … - └── molexp list_projects / materialize_workspace / … + └── molcrafts + packages / open / route + molvis_open / molvis_exec / … + molq_list_jobs / … + molexp_list_projects / … ``` -There is **no** parent server that mounts every provider under `molmcp`. +There is **no** parent server that mounts every provider under `molmcp`, and +**no catalog plane** — routing lives on molcrafts. There is no **harness** +plane either: a harness is a Git SHA of agent tooling, a separate registry from +the `molmcp.providers` entry points that define planes — see +[Harness catalog](harness.md). ## Responsibilities @@ -27,29 +34,27 @@ There is **no** parent server that mounts every provider under `molmcp`. `create_plane(plane_id)` builds one FastMCP server whose **name is the plane id**. Tool names are **bare** (`open`, `list_projects`). Clients see -`molvis__open` / `molexp__list_projects`. Startup **rejects** -`molexp_list_projects` and any `molexp_molexp_*` double-prefix style. +`molcrafts__molvis_open` on the composed server. A focused +`molmcp serve molexp` process still uses bare `list_projects`. Startup +**rejects** registering `molexp_list_projects` on a server named `molexp`. -### 2. Knowledge plane (`molcrafts`) +### 2. Knowledge core (`molcrafts`) -OKF-style pages over the discovery graph: packages → outline → open → compose. +Always on. OKF-style pages over the discovery graph: packages → outline → +open → compose, plus `list_planes` / `route` for optional provider planes. Codegraph ranks are evidence only. Science methods are discovered here and -invoked elsewhere. +invoked elsewhere. `--disable molcrafts` is an error. -### 3. Catalog plane - -Bootstrap only: which planes exist, and which to connect for a free-text task. -Does not run science. - -### 4. Provider planes +### 3. Provider planes `Provider` protocol + `molmcp.providers` entry points. Each provider is its own -plane. Four-condition tool rule still applies (stable signature, read-only -default, high frequency, single-shot). No upstream API mirror. +plane and **can be disabled**. Four-condition tool rule still applies (stable +signature, read-only default, high frequency, single-shot). No upstream API +mirror. ## Request flow (example: draw a molecule) -1. `catalog.route("draw dopamine")` → connect `molvis` (+ ideally `molcrafts`). +1. `molcrafts.route("draw dopamine")` → connect `molvis`. 2. `molcrafts.search` / `open` → real molpy/molvis symbols. 3. `molvis.open` → browser session. 4. `molvis.exec` → agent-written Python (`parse_molecule`, `draw_frame`, …). @@ -64,4 +69,5 @@ default, high frequency, single-shot). No upstream API mirror. - [Provider design](provider-design.md) - [Discovery engine](discovery.md) +- [Harness catalog](harness.md) - [MolVis workbench](../guides/molvis-workbench.md) diff --git a/docs/concepts/harness.example.toml b/docs/concepts/harness.example.toml new file mode 100644 index 0000000..150a7b0 --- /dev/null +++ b/docs/concepts/harness.example.toml @@ -0,0 +1,97 @@ +# Example harness catalog — documentation only. +# +# This file is called harness.example.toml on purpose, and it lives under +# docs/ on purpose. Nothing loads it at run time. The file molmcp actually +# reads is called harness.toml and sits at the root of one published commit +# tree under the cache directory. See docs/concepts/harness.md. +# +# A publication record for this catalog would read: +# +# repo MolCrafts/harness +# sha 9f1c3b2a7d4e0165c8a9b3d27e5f10486c73ab92 +# label official +# +# Neither `sha` nor `label` is a key below, and neither can become one. +# Identity is the commit SHA the caller hands to load_harness_catalog(); a +# label is a note somebody keeps beside that commit. The grammar in +# molmcp.components rejects every key it does not recognise, so writing +# either of them here would stop this file parsing at all. + +# Capability tokens that every piece in this catalog needs from whatever +# process loads it. Two are spellable today — `provider-sdk`, the public +# molmcp.provider_sdk a checkout's provider is written against, and +# `harness-catalog`, this file format. A token outside that pair is a +# grammar error even for a process that would happily support it. +requires = ["provider-sdk", "harness-catalog"] + +# Where this catalog's component paths begin, relative to the checkout root. +# Optional; absent means the checkout root itself, which is what a repository +# purpose-built as a catalog uses. A repository that carries its components +# under a subdirectory names it here so the paths below stay canonical. +component_root = "harness" + +# --------------------------------------------------------------------------- +# Components — one installable piece each. `id` is not written here; it is +# derived as ".", which is why two rows may not share a kind and +# a name. `path` is relative to this file and must start with the directory +# the kind reserves: skills/, agents/, rules/, providers/, overlays/. +# --------------------------------------------------------------------------- + +# A skill is an instruction file an agent reads before it starts working. +[[component]] +kind = "skill" +name = "daily" +path = "skills/daily/SKILL.md" + +# An agent is a definition of one specialised worker the host can dispatch to. +[[component]] +kind = "agent" +name = "librarian" +path = "agents/librarian/AGENT.md" + +# A rule is a constraint that holds across tasks. +[[component]] +kind = "rule" +name = "no-invented-api" +path = "rules/no-invented-api.md" + +# A provider is an MCP plane this checkout contributes. `entrypoint` is a +# "module:object" string naming the class to import; the catalog loader +# stores it and never imports it. Only provider and overlay rows may carry +# one, and both must. +[[component]] +kind = "provider" +name = "bench" +path = "providers/bench/provider.py" +entrypoint = "bench_provider:BenchProvider" + +# An overlay layers domain knowledge onto the code graph discovery builds. +[[component]] +kind = "overlay" +name = "molpy" +path = "overlays/molpy/overlay.py" +entrypoint = "molpy_overlay:MolpyOverlay" + +# --------------------------------------------------------------------------- +# Bundles — named groups of the component ids above. A bundle is written as +# a `component` row whose kind is the literal "bundle"; it is not one of the +# five component kinds and may not be a member of another bundle. Bundle +# names are author-chosen; `daily` and `dev` here are examples, not a +# language-gate pair. +# --------------------------------------------------------------------------- + +[[component]] +kind = "bundle" +name = "sci" +members = ["skill.daily", "rule.no-invented-api", "overlay.molpy"] + +[[component]] +kind = "bundle" +name = "dev" +members = [ + "skill.daily", + "agent.librarian", + "rule.no-invented-api", + "provider.bench", +] +requires = ["provider-sdk"] diff --git a/docs/concepts/harness.md b/docs/concepts/harness.md new file mode 100644 index 0000000..79198e4 --- /dev/null +++ b/docs/concepts/harness.md @@ -0,0 +1,491 @@ +# Harness catalog + +Two different things in MolCrafts are shipped by two different mechanisms, and +the whole point of this page is that they do not touch. + +The first is **molmcp itself**: a Python distribution on PyPI that speaks the +**Model Context Protocol** (MCP) — the wire protocol an AI client such as +Claude Code or Cursor uses to call tools on a server. Its unit of shipping is a +release. Its registry is a **Python entry point**: a line in a package's +`pyproject.toml` that says "when something looks for the `molmcp.providers` +group, hand it this class." That is how a *provider* — one product's MCP +surface, served as its own *plane* (`molvis`, `molq`, `molexp`) — becomes +visible to `molmcp serve`. [Providers](providers.md) covers that path. + +The second is a **harness**: the pile of agent tooling a person or a team +actually works with — instruction files, agent definitions, rules, and +occasionally a plane or a knowledge overlay of their own. A harness is not a +release. It changes several times a week, it belongs to whoever wrote it, and +the interesting question about it is never "which version" but "which exact +commit was I running when that went well?" + +That question is what this page answers. + +## Identity is a Git SHA + +A **Git SHA** is the 40-character lowercase hexadecimal fingerprint Git gives +every commit — `9f1c3b2a7d4e0165c8a9b3d27e5f10486c73ab92`. It is computed from +the commit's content, so it names exactly one tree of files and can never be +made to name a different one. Branch names and tags can: `main` meant something +else last Tuesday, and a tag can be moved. + +A harness is therefore identified by a SHA and by nothing else. There is no +harness version number, no `latest`, and no semantic-versioning range. The +loader enforces this: `molmcp.components.SHA_PATTERN` is `^[0-9a-f]{40}$`, and +`HarnessCatalog` refuses to be constructed with an abbreviated SHA, an +uppercase one, or a branch name. + +The SHA is **not written in the catalog file**. It is passed in by the caller +that already knows which commit it unpacked: + +```python +load_harness_catalog(tree_root, sha, supported_capabilities) +``` + +A file that stated its own SHA could disagree with the tree it sits in — a copy +edited by hand, a rebase, a bad merge — and there would be no way to tell which +of the two was lying. Keeping identity outside the file makes that +disagreement unrepresentable. + +Which SHA an install is running is recorded in an **activation pointer**: a +small JSON file naming three SHAs — `current` (in effect), `staged` (accepted, +waiting), and `previous` (what a rollback would restore). +`molmcp.components.Activation` is the only thing that moves one, and serving +only ever *reads* them. + +There is one such file per **harness source** — one repository this install has +been told it may take a harness from, named in its settings file and described +under [Where a harness comes from](#where-a-harness-comes-from) below. A source +named `official` owns `harness.official.pointer`; a source named `private` owns +`harness.private.pointer` beside it. Both sit in the directory the `cacheDir` +setting names, next to one shared store — `cacheDir/harness` — which is where +the unpacked commit trees themselves live, whichever source activated them. So +each source is activated, and rolled back, on its own, and "which SHA is this +install running?" has one answer per source rather than a single answer for the +install. + +## `official`, `gate`, `canary` are labels on a SHA + +Once identity is a SHA, everything else people want to say about a harness is a +note kept beside one: + +| Label | What it asserts about that SHA | +|-------|--------------------------------| +| `official` | The commit MolCrafts publishes as the default. It is the one the required pull-request check passed on. | +| `gate` | A commit currently under evaluation — accepted for staging, not yet promoted to `current` anywhere but the machine testing it. | +| `canary` | A commit a small number of installs run ahead of everyone else, on purpose, to find out what it breaks. | + +Three properties of these words matter more than their definitions. + +**They are not settings.** `molmcp config set …` has no key for them, and it is +not going to get one. A setting would let two installs disagree about which SHA +is `official` while both believe they are correct; the label belongs to the +commit, not to the reader. + +**They are not environment variables.** molmcp reads no `MOLMCP_*` variable for +anything, and `tests/test_no_env_switches.py` fails the build if a module +starts reading one. Configuration that lives in a single shell cannot be +reported by `molmcp config list`, and two plane servers launched by two clients +would silently disagree about it. + +**They are not keys in the catalog file.** The grammar in +`molmcp.components.catalog` rejects any key it does not recognise, so adding +`label = "official"` to `harness.toml` does not add a label — it stops the file +loading. A label is metadata *about* a commit and a catalog is the contents +*of* one; the loader never sees the label at all. + +No module in `src/` looks any of these three up. They exist so that humans and +CI jobs describing the same commit reach for the same word. (`molmcp gate`, +which checks that this repository's required pull-request check is still wired +the same way in all three places that call it, is unrelated: it validates +molmcp's own CI wiring and knows nothing about harness commits. The label +`official` is named after that check because the check is what earns it.) + +## Two registries, and they are disjoint + +This is the sentence most likely to be undone by a well-meaning future change, +so here it is with its reasons. + +| | MCP planes | Harness plugins | +|---|---|---| +| Authoritative list | the `molmcp.providers` entry-point group | one commit's `harness.toml` | +| Unit | an installed Python distribution | a Git SHA | +| Changes when | somebody releases to PyPI | somebody pushes a commit | +| Discovered by | `importlib.metadata` entry points | reading each activated commit's tree | + +Concretely: none of the following three exists today, and none of them may be +added later without abandoning the split above. + +- **There is no `harness` plane id.** Plane ids are product names (`molcrafts`, + `molvis`, `molq`, `molexp`). "Harness" is a distribution mechanism, not a + product with tools. +- **There is no `molmcp serve harness`.** `molmcp serve` starts the composed + stack; `molmcp serve ` starts one plane for debugging. Neither takes + `harness`, because there is nothing to serve under that name. +- **There is no `molmcp.providers` entry point for a harness.** A harness is + not installed with pip, so it has no `pyproject.toml` for molmcp to read, so + there is nothing for an entry point to point at. + +A harness *may* contribute a plane — that is what a `provider` component is — +but the plane is named by the component's own `name`, and it is mounted for +this process out of the activated tree that declared it. It never becomes an +entry point, and +the catalog id (`provider.bench`) is not the plane id (`bench`); mounting under +the id would namespace its tools as `provider.bench_open`. + +The reason to keep the two lists apart is that they fail differently. An +entry-point plane that breaks was shipped to everyone by a release you can +yank. A harness plane that breaks was a commit one person pushed an hour ago, +and the fix is to move a pointer back. Merging the registries would mean one +recovery procedure for two unrelated failures. + +## What a catalog file says + +A **catalog** is the inventory of one commit: the list of pieces that commit +offers. The tree is never globbed — a file nobody declared in the catalog is +not a component, which is what keeps a stray editor backup out of an agent's +instruction set. + +A catalog holds two kinds of row, and confusingly both are written as +`[[component]]`. The first kind is a component. + +A **component** is one installable piece. There are five kinds of component, +and each one reserves a directory: + +| `kind` | What it is | `path` must start with | `entrypoint` | +|--------|------------|------------------------|--------------| +| `skill` | Instruction file an agent reads | `skills/` | must be absent | +| `agent` | Definition of one specialised worker | `agents/` | must be absent | +| `rule` | A constraint that holds across tasks | `rules/` | must be absent | +| `provider` | An MCP plane this commit contributes | `providers/` | **required** | +| `overlay` | Domain knowledge layered onto the code graph | `overlays/` | **required** | + +An **entrypoint** is a `module:object` string such as +`bench_provider:BenchProvider`. The loader stores it and never imports it — +reading a catalog must not be able to run someone's code. + +The second kind of row is a bundle. A **bundle** is a named group of component +ids, written as a row whose `kind` is the literal string `"bundle"` — which is +why `ComponentKind("bundle")` raises. It is not a sixth component kind, and a +bundle may not contain another bundle. Bundle names are the author's; a +catalog may define none, in which case every component is one implicit +package. `sci` and `dev` are ordinary names, not reserved slots. + +The keys, in full — there are no others, and an unknown one is an error rather +than an ignored line: + +| Where | Keys | +|-------|------| +| top level | `requires` | +| a component row | `kind`, `name`, `path`, `entrypoint` | +| a bundle row | `kind`, `name`, `members`, `requires` | +| derived, never written | `id` — always `"."` | + +`requires` lists **capability tokens**: machinery a piece needs from whatever +process loads it. Two exist today, `provider-sdk` and `harness-catalog`. They +are checked twice, and the two checks are not the same thing. The *language +gate* asks whether the token is even spellable (`ALLOWED_REQUIRES`); an unknown +token is a malformed file. *Eligibility* asks whether this particular process +can honour a spellable token; a token this build does not implement is a +refusal to load, not a malformed file. Keeping them apart is what lets a future +token be added to the grammar without every existing install claiming to +support it. + +## The example file and the file that is read + +This repository publishes exactly one catalog, and it is not a live one: + +| | Published here | Read at run time | +|---|---|---| +| Name | `harness.example.toml` | `harness.toml` | +| Location | `docs/concepts/` | the root of one published commit tree, under `cacheDir` | +| Who reads it | a person, and one test | `Activation.stage` and `create_stack` | + +[`harness.example.toml`](harness.example.toml) is documentation. It is under +`docs/` and never at the repository root, and `tests/test_harness_catalog_fixture.py` +loads it through the real `molmcp.components.load_harness_catalog` so that the +example cannot quietly drift away from the grammar it is illustrating. + +**`harness.toml` is never auto-loaded from the working directory.** The +filename is joined onto a root the caller passes — +`Path(root) / "harness.toml"` in `molmcp/components/catalog.py`, the one place +in `src/` where that name is resolved at all. The only roots molmcp itself ever +passes are the trees of the commits its activation pointers name — one root per +activated source, read in the order the settings file names them. `molmcp serve` +does not look beside itself for a catalog, and neither does `molmcp init`. + +This is the same rule the rest of molmcp follows for `molcrafts.json` and for +the workspace source: a tool that picks up whatever file happens to be next to +the directory you started it in behaves differently for two people running the +same command. + +## Where a harness comes from + +An install names the repositories it may take a harness from in its settings +file, under the key `harness`. The value is an **ordered list of named +sources** rather than a single repository, because one person's tooling is +routinely several: the one MolCrafts publishes, one a team keeps privately, one +that belongs to a particular project. + +```json +{ + "harness": [ + {"name": "official", "locator": "MolCrafts/harness"} + ] +} +``` + +That is a complete `~/.molmcp/settings.json` — the install-wide settings file +described under [Installation](../get-started/installation.md#settings) — with +one source named in it. + +An entry has four keys and no others. `name` is a label you choose; it is how +you refer to the entry, and it is the one key an entry may not leave out. +`owner` and `repo` are the two halves of a GitHub repository path, kept as +separate keys instead of a single `owner/repo` string so that nothing on this +path has to parse one. `ref` is the branch or tag a commit is *resolved from* — +it is not the commit being served, which is the one that entry's own activation +pointer names. + +The three coordinates may be left out while an entry is still being written. An +entry carrying only a `name` loads and is stored exactly as written; what it +cannot do is serve. At serve time an entry that sets some coordinates but not +all of them — setting none of them included — is a configuration error naming +the entry and each field it is missing, rather than a guess. Filling one in +from a default would mean fetching code from a repository nobody asked for. + +**Order is file order, and it is a contract rather than an accident.** Entries +are read first to last as the file writes them, and the first entry that offers +something is the one that answers for it. That is not a promise about some +later release: it is how a piece two sources both ship is settled today, and +[What serving does with the list](#what-serving-does-with-the-list) below is +the whole of the rule. Writing the order down as a contract is what keeps the +answer from coming to depend on the order some dictionary happened to iterate +in. + +**Across settings files, the most specific list replaces the others; it does +not merge.** A project's `.molmcp/settings.json` outranks the user file and +`.molmcp/settings.local.json` outranks both, and the winner's list is the whole +list. That is worth saying out loud, because it is the *opposite* of `excludes`, +`knowledgeScope`, `discoverInclude` and `discoverExclude`, which accumulate +across those same three files. The asymmetry is deliberate: appending a +first-wins list would put the user file's entries at the front and so let the +least specific file outrank the most specific one, which is the inverse of what +every other setting does. + +**There is no built-in default source.** molmcp ships no coordinates for +`MolCrafts/harness` or for anything else, and the entry in the snippet above is +not a fallback that was already there — it is an operator naming a source, the +same act as naming any other. All sources are peers. `official` there is simply +the name chosen for one of them, and the page could as readily have called it +`mine`; the word does mean something, but as a label on a commit, per the table +earlier on this page, and never as a privilege of an entry. (That repository is +also still being stood up: naming a source configures an address, and until the +commit at the far end of it carries a `harness.toml`, there is nothing there to +load.) + +An install whose `harness` key is absent, or is an empty list, simply has no +harness, and serves exactly as it did before any of this existed. That is a +normal configuration, not a degraded one. + +### What serving does with the list + +`molmcp serve` reads the list whole and gives every entry its own turn. A source +that has nothing to contribute costs its neighbours nothing. + +**Each source is activated on its own.** For every entry, in file order, serving +reads that entry's activation pointer — `harness..pointer` under +`cacheDir`, the per-source file introduced [near the top of this +page](#identity-is-a-git-sha) — and serves the commit its `current` names. A +source whose pointer file does not exist yet, or whose pointer activates +nothing, contributes nothing and is **skipped**; the entries around it still +serve. Nothing is fetched and no pointer is written while serving, because +moving a pointer belongs to the commands that were asked to change what is +activated. The one thing that is *not* shrugged off is a pointer naming a +commit whose tree was never unpacked into the store: that stops the serve with +a message naming the source, the SHA and the pointer file, rather than +re-fetching something nobody asked for at start-up. + +**One store, shared by every source.** The unpacked trees all live in the single +`cacheDir/harness` directory; only the pointers multiply. That is a correctness +rule and not a disk-space saving. The store keeps each tree under its SHA alone +and records beside it which repository published that SHA, so a SHA a second +repository lays claim to is refused rather than quietly overwritten: two +repositories cannot both own one commit in one store. Give each source a store +root of its own instead and every tree already published becomes unreachable to +the next source that could have shared it. + +**Two entries may not share a name, compared without regard to case.** +`official` and `Official` look like two entries to a person, but on macOS and +Windows they name one `harness.official.pointer` file, so the second would +silently serve whatever the first activated. Serving stops with a message +naming both spellings. For the same reason a name that cannot be a filename — +one holding a `/` or a `\`, or one shaped like an absolute path — is refused, +naming the entry. Nothing else about a name is prescribed: it is yours to +choose, exactly as an index source's name is. + +**A component two sources both declare is kept once, and the earlier entry +keeps it.** Every activated commit's catalog is read, the components of the kind +being served are collected in source order, and the first source to claim a +given component id — `provider.bench`, `overlay.molpy` — is the one that keeps +it. That collecting-with-a-winner step is a **fold**: several lists become one, +and the rule for a contested key is fixed in advance rather than settled by +whichever list happened to be read last. The displaced declaration is not +served, and it is not silently dropped either: molmcp logs a warning naming the +winning source, the losing source and the contested id, so an operator who did +not intend the overlap learns it from the log rather than from behaviour they +cannot account for. Reordering the list, or dropping the component from one of +the two catalogs, is the whole of the fix — file order is the only priority +control there is, and there is no per-source override. + +For a `provider` component that rule is doing more than tidying up. A component +id is `provider.` and the plane is mounted under the `` half, so two +sources both shipping `provider.demo` are two planes claiming one namespace, and +one of them would be mounted over the other. Keeping the id once is what stops +the pair from mounting twice. + +Bundles are **not** folded across sources. A bundle is a group of ids inside one +catalog, every catalog carries its own `daily` and `dev`, and nothing on the +serving path reads one — what gets served is selected by component kind. Three +activated sources are three catalogs each with its own `daily`, not one merged +`daily`. + +**A pointer file left over from before sources were activated by name is named, +never read.** Such an install has a single `harness.pointer` under `cacheDir` +with no source name in it. molmcp does not read it, and does not migrate it: +when that file is present and no named source has a pointer of its own, molmcp +logs one warning naming the file and serves with no harness at all. +Migration would be machinery for a population that is very nearly empty — +molmcp has no verb that activates a commit yet, so nothing in the product ever +wrote that file, and it can only exist where someone wrote it by hand. Deleting +it, and activating the sources you want under their own names, is the whole +recovery. + +### Authoring an entry, and what to do if you mistype one + +One verb writes the list, and it addresses one origin at a time with a locator: + +```bash +molmcp config harness set MolCrafts/harness --alias official +molmcp config harness set MolCrafts/harness --alias official --enable sci --disable all +molmcp config harness set ~/src/harness --alias local +molmcp config harness remove official +``` + +The locator is a GitHub `owner/repo[@ref]`, a GitHub URL, or a `~/` / absolute +path. `--alias` names the entry (default `origin` on first insert). `--enable` +and `--disable` select catalog bundles on that source; they are not plane +toggles. `molmcp init --enable/--disable` still only mounts provider +planes. A locator already in the list is updated in place; a new origin is +appended **last**. Both subcommands take the same `--project` and `--local` +scope flags as every other `config` write, and with neither they write the +user file. + +They exist because the ordinary write verbs cannot reach this key. `harness` is +a list whose elements are objects, while `config set` and `config add` each take +one string, so both refuse the key outright and answer with the shape of an +entry and the verb that authors one. Reading is unchanged: `molmcp config list` +and `molmcp config get harness` each print the list whole. There is no dotted +path into an individual entry — a dotted read into this key addresses nothing, +and it exits 2 saying so rather than answering `null`, which would have claimed +a coordinate was merely unset. + +Two things this verb deliberately does not do, and you will meet both. + +**A locator that cannot be parsed is refused at set time.** Relative paths, +`http://`, and the `github:` prefix are errors. A GitHub origin without a +reachable checkout is complete enough to store; whether it can fetch is +decided at `harness sync` / `serve`. + +**It cannot repair a settings file that no longer loads.** A settings file is +validated on every *read*, and this verb reads the file before it writes it, +exactly like every other one. An old entry still carrying `owner` / `repo` / +`path` keys is a hard cut: re-run `molmcp config harness set `. +`molmcp config list`, `get`, `set`, `add`, `remove` and `harness`, and +`molmcp serve` itself, all stop with exit status 2 until it is corrected, and +the message names the file and the entry by position, as `harness[0].onwer`. +**The repair is to open that file in an editor.** Nothing is lost and nothing +needs reinstalling — the file is plain JSON and the fix is a text edit. + +## Two repositories, and the older one is leaving + +`MolCrafts/molcrafts-harness` is the **plugin marketplace** MolCrafts used +before this design — a "marketplace" being a repository an agent host is told +about once, from which it then installs plugins by name. It is on its way out. +Nothing in this documentation set offers its URL as a current install address, +and nothing should: an install line for a repository that is being retired is a +promise the maintainers are about to break. + +`MolCrafts/harness` is its replacement in role only. **It is a new, empty +repository — not `molcrafts-harness` renamed.** That distinction is the whole +decision, so it is worth being blunt about why a rename was rejected: + +- A rename carries the old history, and with it the old marketplace layout, the + old plugin manifests, and every stale install instruction anyone ever wrote + down. The new repository's contract is a `harness.toml` at the root of every + commit. Starting from an empty tree makes the first commit that satisfies + that contract also the first commit that exists. +- A rename leaves a redirect. GitHub forwards the old path, so a host still + configured against `molcrafts-harness` keeps working and nobody finds out + they are on the old address until the redirect is removed. +- A rename carries the old licence into the new repository by default, which is + a licensing decision made by accident. See the table below. + +The new repository holds **agent tooling only**: skills, agents, rules, and the +occasional provider or overlay. It is not a monorepo. molq, molexp, molvis and +molpy stay in their own repositories, and moving one into the harness would +make a commit of the harness mean "some agent instructions changed *and* a +science package changed", which is exactly the coupling the SHA-identity model +exists to avoid. + +The old repository is retired only **after** cutover, and retiring it is a +deliberate, separately authorised act. The runbook is +[Retiring the old harness marketplace](../guides/harness-migration.md). + +## Licences + +Three repositories, three separate grants. This table describes them; it does +not change any of them. + +| Repository | Licence | What this page may change | +|------------|---------|---------------------------| +| `MolCrafts/molmcp` — this repository | **BSD-3-Clause**, in [`LICENSE`](https://github.com/MolCrafts/molmcp/blob/master/LICENSE) at the repository root | Nothing. That file is the grant; this row is a description of it. molmcp is not being relicensed. | +| `MolCrafts/molcrafts-harness` — the old marketplace | MIT | Nothing. It keeps the grant it shipped under for as long as it exists. | +| `MolCrafts/harness` — the new catalog repository | Not yet granted; it does not exist yet | Nothing. Its licence is chosen when the repository first exists. | + +Two things follow that are easy to get wrong. + +**A licence is granted once, in the repository it applies to.** Copying +BSD-3-Clause text into `MolCrafts/harness` because molmcp uses it would be a +licensing decision taken as a formatting step. If the new repository ends up +BSD-3-Clause, that must be because someone chose it. + +**A harness commit is not molmcp.** A user's own harness carries whatever +licence its author chose, or none. molmcp loads it; molmcp does not +sub-license it, and nothing in the catalog format asserts anything about the +rights in the tree it describes. + +## Two shapes that were considered and refused + +**A `WikiSkill` as an `init` channel.** `molmcp init ` installs one +managed instruction file — the usage skill in `src/molmcp/skill/SKILL.md` — and +one MCP entry. A proposal to add a second, wiki-shaped skill installed the same +way was rejected. A skill that wraps `packages`, `molvis_open`, `molq_*` or +`molexp_*` in prose is a second copy of the truth about those tools: upstream +renames a tool and the wiki keeps confidently describing the old one. The same +objection retires the chain-of-thought wrapper variant, where the skill narrates +reasoning steps around a call the client can already make directly. +`molmcp init` has exactly one skill channel, and the catalog's `skill` +components are materialised from a checkout, not installed as a second managed +file. + +**A harness plane.** Rehearsed above: no plane id, no `molmcp serve harness`, +no entry point. A harness is where tools come from, not a tool. + +## Read next + +- [Iterate on a harness from a checkout](../guides/iterate-on-a-harness.md) — the three-command loop, starting from a repository on your own disk +- [Retiring the old harness marketplace](../guides/harness-migration.md) — the exit runbook +- [Providers](providers.md) — the other registry, the entry-point one +- [Provider design](provider-design.md) — what earns a tool slot on any plane +- [Installation](../get-started/installation.md#settings) — the settings files the `harness` list is written in, and the other keys beside it diff --git a/docs/concepts/provider-design.md b/docs/concepts/provider-design.md index 61c6d70..b8a9dea 100644 --- a/docs/concepts/provider-design.md +++ b/docs/concepts/provider-design.md @@ -1,8 +1,10 @@ # Provider design contract molmcp is **not** a tool-registration mirror of upstream packages, and it -is **not** a single mega-server. Each provider is its own MCP plane -(`molmcp serve `); clients connect planes on demand. +is **not** a hand-curated mirror of upstream APIs. `molmcp serve` is the +molcrafts core with providers FastMCP-mounted; `molmcp init +--disable ` omits a mount. Each provider still registers as its own +focused FastMCP (`create_plane("molq")`) for tests and debug serve. The primary mechanism for an agent to use a MolCrafts package is the [discovery engine](discovery.md) on the **molcrafts** plane: query the @@ -60,6 +62,23 @@ agents stay out of MCP. | **First-party** (molq, molexp, …) | `src/molmcp/providers//` + entry point `molmcp.providers.`. Upstream package is a **lazy optional** import. Zero FastMCP in the science package. | | **Third-party** | Sibling package or package `mcp` extra — see [Write a Provider](../guides/write-a-provider.md). | +A plane contributed by an activated harness commit comes from neither row: its +registry is a Git SHA, never a `molmcp.providers` entry point, and there is no +plane called `harness` — see [Harness catalog](harness.md). + +**Catalog membership *is* the entry-point group.** `list_planes` and +`known_plane_ids` name exactly what `discover_providers` reported and nothing +else — there is no second list of official names inside `planes.py`. A plane +appears because something registered it on `molmcp.providers`, so every id a +catalog offers is one `molmcp serve` can actually start. `planes.py` still owns +the `purpose` / `when_to_connect` copy for the planes molmcp ships, but that +table is looked up *for* a discovered name, never consulted to produce one; an +unlisted name falls back to a generic sentence, and `tools_hint` is read off +the discovered instance itself. Being in the group settles membership only — +it does not make a plane first-party. In-tree placement +(`src/molmcp/providers//`) still decides that, and the four conditions +above still decide whether any given tool earns a slot. + ## The shape every provider has A provider subclasses `ProviderBase` and declares each tool as a **method** diff --git a/docs/concepts/providers.md b/docs/concepts/providers.md index 1b222d6..1ced2c0 100644 --- a/docs/concepts/providers.md +++ b/docs/concepts/providers.md @@ -6,9 +6,11 @@ stateful runtime data (a job database, an on-disk workspace), a live in-process session, or a capability behind a native extension that source discovery cannot read. -One process serves **one** provider. There is no mega-server and no mounting: -`create_plane("molq")` builds a server named `molq` holding that provider's -tools and nothing else. Passing more than one raises. +`create_plane("molq")` still builds a focused server named `molq` with bare +tools (debug / tests). Default `molmcp serve` uses `create_stack()`: the +molcrafts core **mounts** that server with FastMCP `namespace="molq"`, so +the client sees `molq_list_jobs`. Passing several providers to +`create_plane` still raises — composition is `create_stack`. > **Read [provider-design.md](provider-design.md) first.** It defines the > conditions a tool must satisfy before earning a slot. Most ideas for new @@ -77,6 +79,11 @@ A provider reaches molmcp through the `molmcp.providers` entry-point group: molq = "molmcp.providers.molq:MolqProvider" ``` +This entry-point group is the authoritative list of planes, and it is a +different registry from the harness catalog, which is identified by a Git SHA: +there is no harness entry point and no `molmcp serve harness` — see +[Harness catalog](harness.md). + `molmcp serve molq` loads the entry point whose name matches the plane id and serves that provider alone. **Every provider is instantiated with `cls()`** — no arguments. Anything an operator must be able to change therefore belongs in diff --git a/docs/get-started/deploy.md b/docs/get-started/deploy.md index 5c42d54..80e336f 100644 --- a/docs/get-started/deploy.md +++ b/docs/get-started/deploy.md @@ -1,25 +1,20 @@ # Deploy locally (stdio) -Local **stdio** MCP: the client spawns `molmcp serve ` as a subprocess -per session. No HTTP, no shared mega-server — **one plane per connection**. +Local **stdio** MCP: the host spawns **one** `molmcp serve` subprocess. +Providers are FastMCP-mounted onto the molcrafts core. --- ## What molmcp serves -| Plane | Command | What the agent sees | -|-------|---------|---------------------| -| **catalog** | `molmcp serve catalog` | `list_planes`, `route(task)` — bootstrap only | -| **molcrafts** | `molmcp serve molcrafts` | Knowledge pages: `packages`, `outline`, `open`, `search`, `compose`, … | -| **molvis** | `molmcp serve molvis` | Live stage session: `open`, `exec`, `poll_events`, … | -| **molq** | `molmcp serve molq` | Job store (+ opt-in submit/cancel when enabled) | -| **molexp** | `molmcp serve molexp` | Workspace layout / scaffold tools | +| Command | What the agent sees | +|---------|---------------------| +| **`molmcp serve`** | Core: `list_planes`, `route`, `packages`, `outline`, `open`, … plus namespaced mounts `molvis_open`, `molq_list_jobs`, `molexp_list_projects`, … | +| **`molmcp serve molvis`** (debug) | Vis-only process, bare `open` / `exec` | -Connect only the planes the session needs. Tool ids are -`__` (MCP server name + bare tool name). - -There is no parent `python -m molmcp` that mounts every provider under one -server name. +`molcrafts` cannot be disabled. `molmcp init grok --disable molq` omits that +mount. Tool ids on the composed server are `molcrafts__packages` and +`molcrafts__molvis_open`. ## Prerequisites @@ -52,7 +47,6 @@ server name. ### Claude Code ```bash -claude mcp add catalog -- molmcp serve catalog claude mcp add molcrafts -- molmcp serve molcrafts claude mcp add molvis -- molmcp serve molvis # optional claude mcp list @@ -63,10 +57,6 @@ claude mcp list ```json { "mcpServers": { - "catalog": { - "command": "uv", - "args": ["run", "--directory", "/path/to/molmcp", "molmcp", "serve", "catalog"] - }, "molcrafts": { "command": "uv", "args": ["run", "--directory", "/path/to/molmcp", "molmcp", "serve", "molcrafts"] @@ -77,7 +67,7 @@ claude mcp list ## Recommended agent loop -1. `catalog.route("…")` → which planes to connect. +1. `molcrafts.route("…")` → which optional provider planes to connect. 2. `molcrafts.packages` / `outline` / `open` → real APIs into context. 3. Call science from agent Python (or `molvis.exec` for a live canvas). 4. Never invent MCP tools that re-export molpy/molrs methods. diff --git a/docs/get-started/installation.md b/docs/get-started/installation.md index 2a22454..b99e3cd 100644 --- a/docs/get-started/installation.md +++ b/docs/get-started/installation.md @@ -13,25 +13,33 @@ pip install molcrafts-molmcp ```bash uv add --prerelease=allow molcrafts-molmcp +# or, into the active environment: +uv pip install --prerelease=allow --upgrade molcrafts-molmcp ``` -!!! note "Why the flag" +Check the binary you actually got: - molmcp requires **FastMCP 4** for MCP 2026-07-28, and FastMCP 4 is still - in beta — PyPI's 4.x line is `4.0.0b2` with no final release yet. pip - installs it without ceremony, but uv does not enable pre-releases for a - dependency of a dependency, so it reports: +```bash +which molmcp +molmcp --version +``` + +!!! warning "Without `--prerelease=allow`, uv will not install 0.6+" + + molmcp requires **FastMCP 4** (MCP 2026-07-28). FastMCP 4 is still beta + (`4.0.0b5`). `pip install -U molcrafts-molmcp` is fine; uv is not: ``` - Because only fastmcp<4.0.0b1 is available and molcrafts-molmcp - depends on fastmcp>=4.0.0b1 ... cannot be used. + Because only fastmcp<4.0.0b5 is available and molcrafts-molmcp + depends on fastmcp>=4.0.0b5 ... cannot be used. ``` - Pinning an exact `==4.0.0b2` does not help — uv refuses that for the same - reason. FastMCP 3.x is not an alternative: it speaks the older protocol, - and molmcp's planes are built on the new one. + Bare `uv pip install --upgrade molcrafts-molmcp` can also **downgrade** + to 0.2.1 (the last release whose dependencies are all stable). Always + pass `--prerelease=allow` until FastMCP 4.0.0 final ships. - The flag stops being necessary the day FastMCP 4.0.0 ships. + `--version` exists from **0.6.1**. An older CLI prints + `the following arguments are required: command` instead. ## What gets installed @@ -115,12 +123,29 @@ molmcp config set sources.atomiverse pkg:atomiverse | `maxCacheAgeDays` | Retention window for extraction payloads (default 30) | | `pythonEnv` | Environment to discover from: a venv root, a python, or a site-packages dir | | `discoverInclude` / `discoverExclude` | Force a distribution in or out of auto-discovery | +| `harness` | Ordered list of named harness sources, each an object `{name, owner, repo, ref}` | | `molexp.workspace` | Default molexp workspace path | | `molq.database` | Override the molq job database | Unknown keys are rejected. A mistyped `indexWorkspaces` that quietly does nothing is worse than one that says so. +`harness` is the one key in that table whose elements are objects, so the +string-valued write verbs cannot author it and it has two subcommands of its own: +`molmcp config harness set MolCrafts/harness [--alias NAME] [--enable BUNDLE] [--disable BUNDLE]` +upserts one entry, `molmcp config harness remove --name NAME` drops one, and both +take the same `--project` / `--local` scope flags as the verbs above. What the +list is for, what an entry means, what a half-written one does at serve time, +and a worked snippet of the file live on +[Harness catalog](../concepts/harness.md); molmcp ships no default source, so an +install that names none simply has no harness. + +Because rejection happens on every *read*, and every `config` verb reads the +file before it writes it, a typo anywhere in the file stops all of +`config list`, `get`, `set`, `add`, `remove` and `harness` — and `molmcp serve` +too — with exit status 2, the message naming the file and the offending key. +The fix is to edit that same file; no verb can do it for you. + ### `molcrafts.json` Still accepted with an explicit `--config PATH`, but no longer picked up from @@ -129,6 +154,7 @@ was. ## Next steps -- **[Quickstart](quickstart.md)** — serve catalog + molcrafts and wire a client -- **[Architecture](../concepts/architecture.md)** — one plane per connection -- **[Deploy](deploy.md)** — multi-link stdio layout for Claude Code +- **[Quickstart](quickstart.md)** — `molmcp serve` and `molmcp init` +- **[Architecture](../concepts/architecture.md)** — FastMCP composition +- **[Harness catalog](../concepts/harness.md)** — the ordered `harness` source list, how to write one into your settings file, and why a harness is a Git SHA rather than a plane +- **[Deploy](deploy.md)** — local stdio for Claude Code diff --git a/docs/get-started/migrating-from-0.2.md b/docs/get-started/migrating-from-0.2.md index 3ec1f74..a9a894c 100644 --- a/docs/get-started/migrating-from-0.2.md +++ b/docs/get-started/migrating-from-0.2.md @@ -15,22 +15,26 @@ noted. ships. See [Installation](installation.md#with-uv) for why; pip needs nothing extra. -## One server became five +## One server became a core plus optional planes Before, `molmcp serve` started one process that mounted every provider and -prefixed their tools. Now each product domain is its own MCP connection: +prefixed their tools. Now **`molcrafts` is the always-on core** (knowledge +plus `list_planes` / `route`), and each provider is its own MCP connection: | Plane | Serves | |---|---| -| `catalog` | Which planes exist and which to route to | -| `molcrafts` | Knowledge and discovery over installed packages | +| `molcrafts` (core) | Knowledge pages and routing; cannot be disabled | | `molq` | Job lifecycle | | `molexp` | Experiment-data workspaces | | `molvis` | A live viewer session | -A client connects to the planes it wants, as separate servers. There is no -mega-server to fall back to: passing more than one provider raises -`ValueError: multi-provider servers are removed; serve one plane per process`. +A short-lived `catalog` plane existed in 0.5 and has been absorbed: those +tools live on molcrafts. `molmcp serve catalog` errors. Provider planes +are the only `--disable` targets. + +`molmcp serve` now FastMCP-mounts providers onto molcrafts. Passing more +than one provider to `create_plane` still raises; composition is +`create_stack()`. ## Every tool id changed @@ -43,8 +47,8 @@ set — nothing silently keeps working: | `mcp__molmcp__molcrafts_packages` | `mcp__molcrafts__packages` | | `mcp__molmcp__molcrafts_search` | `mcp__molcrafts__search` | | `mcp__molmcp__molcrafts_open` | `mcp__molcrafts__open` | -| `mcp__molmcp__molvis_open` | `mcp__molvis__open` | -| `mcp__molmcp__molq_list_jobs` | `mcp__molq__list_jobs` | +| `mcp__molmcp__molvis_open` | `mcp__molcrafts__molvis_open` (composed) | +| `mcp__molmcp__molq_list_jobs` | `mcp__molcrafts__molq_list_jobs` (composed) | Any prompt, allowlist, or auto-approve rule naming a tool must be rewritten. @@ -68,7 +72,7 @@ fails at startup with an argparse error rather than serving anything: } ``` -`molmcp client ` generates this for you and omits planes whose package is +`molmcp init ` generates this for you and omits planes whose package is not installed. Bare `molmcp` no longer starts a server either — it prints the plane catalog. diff --git a/docs/get-started/quickstart.md b/docs/get-started/quickstart.md index bfa2d13..c318122 100644 --- a/docs/get-started/quickstart.md +++ b/docs/get-started/quickstart.md @@ -1,7 +1,8 @@ # Quickstart -Stand up **on-demand multi-plane** MCP for MolCrafts: connect only the product -domains you need. There is no single process that mounts every tool. +Stand up MolCrafts MCP: **`molmcp serve`** is the knowledge core with +enabled providers FastMCP-mounted (`molvis_open`, …). `molmcp init ` +writes that one MCP entry and the managed skills (`molcrafts`, `molexp-plan`). ## 1. List planes @@ -11,56 +12,47 @@ molmcp planes molmcp route "draw dopamine" ``` -Built-in planes include `catalog` (routing) and `molcrafts` (knowledge pages). -Provider planes (`molvis`, `molq`, `molexp`, …) appear when their packages / -entry points are available. +`molcrafts` is the core connection. Provider planes appear when their +packages / entry points are available. `--disable molcrafts` is an error; +`--disable molvis` (and the other providers) is the supported toggle. -## 2. Serve one plane per process +## 2. Serve ```bash -# Terminal A — bootstrap routing -molmcp serve catalog +# Composed core + provider mounts (needs at least one configured source) +molmcp serve -# Terminal B — knowledge pages (needs at least one configured source) -molmcp serve molcrafts - -# Terminal C — live viewer (optional) +# Debug one provider only (bare tool names) molmcp serve molvis ``` -Each process is one MCP server whose **name is the plane id**. Clients see -bare tool names under that server: `molcrafts__packages`, `molvis__open`, -`catalog__route`. - -## 3. Connect from Claude Code (multi-link) +On the composed server, clients see `molcrafts__packages` and +`molcrafts__molvis_open`. -Register **one MCP entry per plane**: +## 3. Connect from Claude Code ```bash -claude mcp add catalog -- molmcp serve catalog -claude mcp add molcrafts -- molmcp serve molcrafts -# only when drawing: -claude mcp add molvis -- molmcp serve molvis +molmcp init claude +# or: +claude mcp add molcrafts -- molmcp serve ``` -JSON shape (any client that supports multiple servers): +JSON shape: ```json { "mcpServers": { - "catalog": { - "command": "molmcp", - "args": ["serve", "catalog"] - }, "molcrafts": { "command": "molmcp", - "args": ["serve", "molcrafts"] + "args": ["serve"] } } } ``` Use absolute paths / `uv run --directory …` if the client’s PATH is thin. +`molmcp init grok` writes this map (one composed `serve`) and the managed +skills (`molcrafts`, `molexp-plan`); drop mounts with `--disable`. ## 4. Knowledge plane tools @@ -68,6 +60,7 @@ On **molcrafts**, the main path is hierarchical pages: | Tool | Role | |------|------| +| `list_planes` / `route` | Optional provider planes to connect | | `packages` | L0 package directory — choose sources | | `outline` | Module / symbol map for one source | | `open` | Inject one symbol page (optional source body) | @@ -76,7 +69,7 @@ On **molcrafts**, the main path is hierarchical pages: | `info` | Ops / health — not the primary discovery path | Science methods are **discovered** here and **invoked** in agent Python or -inside `molvis` `exec` — they are never re-wrapped as MCP science tools. +via `molvis_exec` — they are never re-wrapped as MCP science tools. ## 5. HTTP instead of stdio @@ -89,6 +82,6 @@ Non-loopback HTTP requires auth configuration — see [Deploy](deploy.md). ## What's next? - **[Deploy](deploy.md)** — full local stdio layout and client wiring -- **[Architecture](../concepts/architecture.md)** — plane model +- **[Architecture](../concepts/architecture.md)** — core + provider planes - **[MolVis workbench](../guides/molvis-workbench.md)** — open / exec / poll_events - **[Write a Provider](../guides/write-a-provider.md)** — add a product plane diff --git a/docs/guides/adopt-a-data-directory.md b/docs/guides/adopt-a-data-directory.md index 294d793..675bc4e 100644 --- a/docs/guides/adopt-a-data-directory.md +++ b/docs/guides/adopt-a-data-directory.md @@ -80,7 +80,8 @@ per-engine text. run_adoption(..., ingest=["lammps_log", "tensorboard"]) ``` -Omit `ingest` and nothing is converted — `metrics/metrics.jsonl` is +Omit `ingest` and nothing is converted — the metrics surface (`metrics/zarr/` +dense SoT + optional `metrics/metrics.jsonl` WAL) is append-only, so ingestion is never implied by silence. The converters are molexp's own (`molexp.plugins.metrics_ingest`): LAMMPS thermo through molpy's log reader, tfevents through molexp's TensorBoard plugin, CSV through stdlib diff --git a/docs/guides/harness-migration.md b/docs/guides/harness-migration.md new file mode 100644 index 0000000..1cae16e --- /dev/null +++ b/docs/guides/harness-migration.md @@ -0,0 +1,111 @@ +# Retiring the old harness marketplace + +`MolCrafts/molcrafts-harness` is the plugin **marketplace** MolCrafts used +before harness identity became a Git SHA — a marketplace being a repository an +agent host is told about once, after which it installs plugins from it by name. +It is being retired. Its replacement is `MolCrafts/harness`, a **new and empty +repository**, and the difference between "new and empty" and "the old one +renamed" is the reason this runbook exists at all. +[Harness catalog](../concepts/harness.md) is the concept page; read it first if +the words *plane*, *catalog* or *SHA* are not already familiar. + +This page is a **runbook for a human**, not a script. It has five steps, and it +ends. The five steps are the ones that can be done inside this repository, with +a diff a reviewer can read. Everything that mutates a remote repository on +GitHub is listed after the stop, as work that needs its own authorisation. + +Nothing here calls `gh`. + +## 1. Inventory every sentence that still points a reader at the old repository + +Search the whole working tree — public documentation, internal notes, the usage +skill, agent-facing hint and error strings — for `molcrafts-harness`, and read +each hit in context. Classify each one: + +- **An install instruction**, telling someone to add that repository to their + host right now. These are wrong and must go. A host wired to a repository + that is about to disappear fails at the moment its user is least able to + diagnose it, and an agent that was told the address will repeat it. +- **A historical mention**, naming the repository as the thing being replaced. + These are fine and this page is one of them. + +Pin the outcome rather than trusting the search: `tests/test_harness_catalog_fixture.py` +scans `docs/` and `.claude/notes/` and fails if the old address ever reappears +as a current `marketplace add`. A one-off grep proves the tree is clean today; +the test is what keeps it clean after the next writer forgets. + +## 2. Write down the two-repository decision where it will be found again + +Two facts have to survive longer than anyone's memory of this migration: + +- `MolCrafts/harness` is a **new empty repository**. It is not + `molcrafts-harness` under a different name, and the old repository's history + is not carried into it. +- **Identity is a Git SHA.** Not a version, not a tag, not a branch. + +Both go in `.claude/notes/harness-contract.md`, which holds those two rules and +nothing else, indexed from `.claude/notes/README.md`. Keeping it to two rules is +deliberate: a note that also restates the catalog keys becomes a second copy of +the grammar, and the copy is the one that goes stale. The keys live on the +concept page beside the example that demonstrates them. + +## 3. Publish the example catalog, and say which filename is actually read + +`docs/concepts/harness.example.toml` is the published example. The file a +consumer reads is `harness.toml`, at the root of one published commit tree. +Both facts have to be written down together, because a reader who sees only the +first will reasonably assume the example is the live file and start editing it. + +Two placement rules follow, and both are load-bearing: + +- The example stays under `docs/`. At the repository root it would sit exactly + where a future loader might look for a real catalog, and molmcp's own + repository would be the first thing to load it. +- Nothing auto-loads it, or any catalog, from the working directory. The + filename is joined onto a root the caller passes in — one place in `src/`, + `molmcp/components/catalog.py` — and the only roots molmcp passes are the + trees of the commits its activation pointers name, one per activated source, + in the order the settings file lists them. + +## 4. Put the licence table on the concept page + +Three repositories, three separate grants, one table on +[Harness catalog](../concepts/harness.md#licences). The table describes the +grants; it does not issue them. + +- `MolCrafts/molmcp` is **BSD-3-Clause**, and the authority for that is the + `LICENSE` file at this repository's root. This migration does not relicense + molmcp, and no row in that table can. +- `MolCrafts/molcrafts-harness` is MIT and stays MIT for as long as it exists. +- `MolCrafts/harness` has no licence yet, because it has no commits yet. Its + grant is settled when the repository first exists, by somebody choosing it. + Reproducing molmcp's BSD-3-Clause text there because it was nearby would be a + licensing decision taken as a formatting step. + +## 5. STOP + +The runbook ends here. What remains is a set of operations against remote +repositories on GitHub, and **each of them needs its own authorisation before +anybody runs it.** They are described below so the shape of the remaining work +is clear — the descriptions are not instructions to act now, and no command on +this page is meant to be pasted into a shell: + +- **Creating `MolCrafts/harness`** (`gh repo create`) — a new empty repository, + with its licence chosen at that moment rather than inherited. +- **Archiving or deleting `MolCrafts/molcrafts-harness`** — only *after* + cutover, and only once no host configuration still points at it. Archiving + leaves the history readable; deleting does not, and deleting also frees the + name for anyone to take. +- **Bundling the old history** (`git bundle`) — if any of it is worth keeping, + it is captured before either of the above, not after. + +One thing is out of scope even with authorisation: **do not pile provider +repositories into the new one.** molq, molexp, molvis and molpy keep their own +repositories. A harness commit means "the agent tooling changed"; if a science +package can also change under the same SHA, the SHA stops answering the one +question it exists to answer. + +## Read next + +- [Harness catalog](../concepts/harness.md) — SHA identity, the two registries, the licence table +- [Providers](../concepts/providers.md) — the other registry, the `molmcp.providers` entry-point one diff --git a/docs/guides/iterate-on-a-harness.md b/docs/guides/iterate-on-a-harness.md new file mode 100644 index 0000000..bf8dfbb --- /dev/null +++ b/docs/guides/iterate-on-a-harness.md @@ -0,0 +1,279 @@ +# Iterate on a harness from a checkout + +A **harness** is the pile of agent tooling you actually work with: instruction +files an AI coding agent reads before it starts (**skills**), definitions of +specialised workers it can delegate to (**agents**), and constraints that hold +across every task (**rules**). If you keep those in a git repository, molmcp can +install them into your AI client for you — and, more to the point, remember +exactly which commit it installed. + +This guide starts where most people actually are: a checkout on your own disk, +pushed nowhere. It gets those files into your client, then takes you round the +loop again after you change one. Three commands, and the rest of this page is +what each of them is for: + +```bash +molmcp config harness set /abs/path/to/checkout --alias local +molmcp harness sync local +molmcp init claude +``` + +The first names the checkout. The second pins it to one commit. The third +installs what that commit declares. [Harness catalog](../concepts/harness.md) is +the full contract behind all three; everything needed to run the loop is here. + +Two words before the first step. A **host** is the AI client being wired — +`claude`, `cursor`, `codex` or `grok` — each of which keeps its files in its own +directory under your home. A **catalog** is a file in your checkout listing what +that repository offers. Nothing is installed without one, because molmcp never +walks your tree looking for likely files: an editor backup sitting next to a +skill is not a skill, and the way molmcp knows that is that you did not list it. + +## 1. Write the catalog + +The catalog is `harness.toml`, and it sits at the **root of the checkout** — +beside `.git`, not inside a subdirectory. It has to be in every commit you +intend to install from, for the reason step 3 explains: what molmcp reads is a +commit, not your working directory. + +Here is one that loads: + +```toml +component_root = "plugins/mol" + +[[component]] +kind = "skill" +name = "spec" +path = "skills/spec/SKILL.md" + +[[component]] +kind = "agent" +name = "architect" +path = "agents/architect.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.spec"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["agent.architect"] +``` + +Every row is spelled `[[component]]`, which is TOML's syntax for "another +element of a list of tables". The first two rows above are **components** — one +installable piece each — and the last two are **bundles**, which are something +else entirely; they are covered at the end of this step. + +A component row carries a `kind`, a `name`, and a `path`. There are five kinds, +and each one reserves a directory that the `path` must start with: + +| `kind` | What it is | `path` starts with | +|--------|------------|--------------------| +| `skill` | Instruction file an agent reads | `skills/` | +| `agent` | Definition of one specialised worker | `agents/` | +| `rule` | A constraint that holds across tasks | `rules/` | +| `provider` | An MCP server this commit contributes | `providers/` | +| `overlay` | Domain knowledge layered onto the code graph | `overlays/` | + +The prefix is not decoration and it is not inferred: a `skill` row whose path +does not begin `skills/` stops the file loading. The first three kinds are the +ones that become files in a host, and they are what this guide follows; the +[concept page](../concepts/harness.md#what-a-catalog-file-says) covers the other +two, which additionally need an `entrypoint`. (MCP is the Model Context +Protocol, the wire protocol an AI client speaks to call tools on a server; a +`provider` row is a server of that kind, contributed by the commit itself.) + +`component_root` is optional, and it names the directory the component tree +begins at. The example above is for a repository that keeps its tooling under +`plugins/mol/`, so `skills/spec/SKILL.md` is read from +`plugins/mol/skills/spec/SKILL.md`. A repository laid out for this purpose — one +whose `skills/` and `agents/` sit at the top — simply leaves the key out. Note +what it does *not* move: `harness.toml` itself is always at the checkout root, +whatever `component_root` says. + +You never write an id. A row's id is derived as `.`, which is why +the bundle above refers to `skill.spec` and not to `spec`. + +**The `daily` and `dev` bundles are required by the grammar.** A catalog missing +either one does not load, so the two rows above are the minimum. Be clear about +what you are getting for them: nothing in molmcp reads a bundle today. They are +declared, they are validated — every member must be the id of a component in the +same file — and no command consults them. Write them, and do not go looking for +their effect. + +Now commit the file, along with whatever it points at. + +## 2. Name the checkout as a source + +A **harness source** is one repository this install is allowed to take a harness +from. You give it a name of your choosing and one origin: + +```bash +molmcp config harness set /abs/path/to/checkout --alias local +``` + +``` +wrote ~/.molmcp/settings.json +``` + +The command also prints the settings file back as JSON, so you can see the entry +it wrote. `local` there is a label, not a keyword — it is how you will refer to +this source in every later command, and you could as easily have called it +`mine`. + +**The path must be absolute, or start with `~/`.** A path like `./checkout` is +refused, with a message that says why: your settings file is shared by every +project on the machine, and a molmcp server started by an AI client inherits +whatever working directory that client happened to be in, so one stored +`./checkout` would name a different repository in every session. `~/harness` is fine, because home is the same +directory in every session. The refusal fires when something tries to *use* the +entry — the sync in the next step, or `molmcp serve` starting a server — rather +than when you write it, so an entry can be half-authored across several edits +without anything breaking in between. + +An entry names **one** origin. `--path` is a checkout on disk; `--owner`, +`--repo` and `--ref` are a GitHub repository. The two shapes are mutually +exclusive, and putting them on the same entry is refused at the moment you write +it, with the file left as it was. That is the whole of the local-versus-remote +decision: there is no flag anywhere later that switches between them, because +the entry's shape already answers the question. + +## 3. Sync: pin the source to a commit + +Configuring a source fetches nothing. `sync` is the verb in between: + +```bash +molmcp harness sync local +``` + +``` +local: 11653d848fbdec2b54c744e4c922a474819e1403 activated + tree ~/.cache/molmcp/discovery/harness/commits/11653d84.../tree + pointer ~/.cache/molmcp/discovery/harness.local.pointer +``` + +Three things happened, and each corresponds to a line. + +The 40-character string is a **Git SHA**, the fingerprint git computes for every +commit from its own content. It names exactly one tree of files and can never be +made to name a different one, which is why it — and not a version number, a tag +or a branch name — is what a harness is identified by here. For a local source +the SHA is whatever `HEAD` resolves to in your checkout: the tip of the branch +you have checked out right now. + +The `tree` line is where that commit was unpacked, in a store shared by every +source. (The directory is named by the full SHA; it is shortened above to fit.) + +The `pointer` line is this source's **activation pointer**: a small JSON file +recording which SHA is in effect. Each source owns one, named after it, so +`local` and a second source called `official` are activated independently and +neither can move the other's. + +Two consequences are worth stating plainly, because they are the reason to do +any of this instead of copying files by hand. + +**What is published is a commit, never your working tree.** The commit's tree is +read out with `git archive` at the resolved SHA, so a file you edited and did +not commit is simply absent from it, and a file you created and did not `git add` +does not exist as far as molmcp is concerned. This is not friction to work +around — it is the feature. "Which harness was I running when that session went +well?" has an answer only if the thing being installed was a commit. + +**Syncing the same commit twice is one sync.** The second run reports `already +activated` and leaves the pointer alone, deliberately: a pointer also records +the SHA it displaced, and re-activating the commit that is already current would +overwrite that record with the SHA that is already current. + +## 4. Install into the host + +Sync moved a pointer; it wrote nothing into your client. `init` is what reads +the pointer and installs what that commit's catalog declares: + +```bash +molmcp init claude +``` + +``` +wrote ~/.claude/skills/molcrafts/SKILL.md +placed 5 harness catalog component file(s), 0 refused +``` + +(Those are the two lines this loop is about. The rest of the output reports the +client's MCP configuration and the older `--source` route named at the end of +this page, neither of which is involved here.) + +The checkout behind that run declared five component rows — three skills, an +agent and a rule — and here is where they landed: + +``` +~/.claude/skills/spec/SKILL.md +~/.claude/skills/impl/SKILL.md +~/.claude/skills/review/SKILL.md +~/.claude/agents/architect.md +~/.claude/rules/design-principles.md +``` + +The mapping is one substitution. Each kind has a directory in the host — +`skills/`, `agents/` and `rules/` under `~/.claude` for this host, and the same +three under `~/.cursor`, `~/.codex` or `~/.grok` for the others — and the kind's +prefix in the catalog path is replaced by it. So `skills/spec/SKILL.md` in the +catalog becomes `~/.claude/skills/spec/SKILL.md` in the host: the layout you keep +in the repository is the layout you get. + +`0 refused` counts declared rows that were deliberately not installed, and there +are exactly two ways to earn one. A `provider` or `overlay` row is refused with +*kind has no host destination*: a provider is a server molmcp mounts and an +overlay is knowledge molmcp's own code index reads, so neither is a file any +client keeps. A row aiming at `skills/molcrafts/` is refused with *managed usage +skill is owned by molmcp init* — that directory holds the instruction file molmcp +writes for itself, the first line of the output above, and no catalog may take it +however the path is spelled. + +One failure will find you early. Declaring a component whose file you forgot to +commit passes `sync` cleanly — a catalog is checked as a *file*, and nothing +confirms that the paths in it exist — and then fails `init`, naming the +component id and the path it could not find. Nothing is written when that +happens: every source is checked before the first byte is copied, so you do not +get half a harness. Commit the missing file and run `init` again. + +## 5. Go round again + +Changing a skill is the same loop, and it is short: + +```bash +git commit -am "sharpen the spec skill" +molmcp harness sync local +molmcp init claude +``` + +`sync` resolves `HEAD` again, finds a new SHA, publishes it, and moves the +pointer. The SHA that was current becomes the pointer's `previous`, and both +trees stay in the store — the new commit's and the one it displaced — so both +remain readable. `init` then copies the new files over the old ones. + +The step people leave out is the commit, and it is the one step that cannot be +skipped: an edit you have not committed is not in any commit, so `sync` will +resolve the same SHA as last time and report `already activated`. When the loop +seems to do nothing, that is almost always what happened. + +## Several sources, and the ones you have not synced + +An install may name more than one source — the one you are writing, one your +team keeps, one that belongs to a project — and they are read in the order the +settings file lists them. + +You do not have to sync all of them. **A configured source that has never been +synced contributes nothing and is not an error**: it has no activation pointer, +so `molmcp init` passes over it and installs the others. Configuring a source is +naming an address; syncing it is the separate act of deciding to run it. A new +install that has configured sources and synced none of them is in an ordinary +state, not a broken one. + +## Read next + +- [Harness catalog](../concepts/harness.md) — SHA identity, the full catalog grammar, and what serving does with a list of sources +- [CLI reference](../reference/cli.md#molmcp-harness) — every flag on `molmcp config harness` and `molmcp harness sync` +- [Installation](../get-started/installation.md#settings) — the settings files a harness source is written into, and the other keys beside it diff --git a/docs/guides/molvis-workbench.md b/docs/guides/molvis-workbench.md index a697087..8cde5bd 100644 --- a/docs/guides/molvis-workbench.md +++ b/docs/guides/molvis-workbench.md @@ -12,10 +12,13 @@ Two wills act on one session and neither blocks the other. The human's "change * ## The loop -Connect the **molvis** plane (`molmcp serve molvis`). Tool ids are -`molvis__open`, `molvis__exec`, … (server name + bare tool). +Default `molmcp serve` mounts molvis onto the molcrafts core. Tool ids are +`molvis_open`, `molvis_exec`, `molvis_poll_events` (FastMCP namespace). +A debug `molmcp serve molvis` process still uses bare `open` / `exec`. -`open` → `exec` (build and draw) → the human looks and clicks → `poll_events` → `exec` (read the selection, edit, redraw) → `close`. +`molvis_open` → `molvis_exec` (build and draw) → the human looks and clicks → +`molvis_poll_events` → `molvis_exec` (read the selection, edit, redraw) → +`molvis_close`. Step one, once `open` has returned and the user has the viewer open in a browser: build the molecule and put it on the canvas. `stage` is already bound in the namespace; nothing else is imported for you. @@ -72,6 +75,8 @@ The full aspirin rehearsal — start the server, open, build, look, click, poll, The reason is honesty about what the thing is. An interactive dialogue script that needs a person to click a benzene ring is neither a runnable product example nor a CI test, and filing it as one advertises a guarantee no maintainer can keep. In-tree tests pin the workbench mechanics only: session lifecycle, namespace persistence, journal ordering under concurrent writes, and one round trip against real molvis over its in-process transport, no browser involved. +**One word, two meanings.** The `molvis-agent-e2e/` playbook is a *test* harness in the ordinary English sense — a rig you drive a system with — and it is **not** the Git SHA plugin catalog documented in [Harness catalog](../concepts/harness.md). That other harness is a repository of agent tooling (skills, agents, rules, occasionally a plane) pinned by commit; this one is a directory of dialogue scripts, has no `harness.toml`, is registered nowhere, and is never activated by molmcp. Nothing in this section is an instruction to put the playbook in a harness commit. + ## Read next - **[Provider design](../concepts/provider-design.md)** — the primitives, the no-invented-API rule, and the local trust model diff --git a/docs/guides/write-a-provider.md b/docs/guides/write-a-provider.md index 553d550..ca94bc9 100644 --- a/docs/guides/write-a-provider.md +++ b/docs/guides/write-a-provider.md @@ -150,6 +150,11 @@ molpack = "molpack_mcp:MolpackProvider" The key (`molpack` here) is just a label — molmcp doesn't use it. The value is the dotted path to your Provider class. +This entry point is how an *installed* provider is found. A provider shipped +inside a harness commit is registered the other way — by a `[[component]]` row +in that commit's catalog, identified by a Git SHA — and never gets an entry +point or a plane named `harness`; see [Harness catalog](../concepts/harness.md). + ## Step 5 — Test it ```python @@ -196,7 +201,7 @@ plane also indexes it — its symbols are reachable through `molcrafts` To wire into a client: ```bash -molmcp client # every plane, as standard mcpServers JSON +molmcp init grok # managed skills + composed molmcp serve claude mcp add molpack -- molmcp serve molpack ``` diff --git a/docs/index.md b/docs/index.md index f41e33b..666390c 100644 --- a/docs/index.md +++ b/docs/index.md @@ -1,13 +1,13 @@ --- title: molmcp -description: Multi-plane MCP for MolCrafts — one product domain per connection, knowledge pages on demand, no mega-server. +description: MolCrafts MCP — one composed serve (knowledge core + namespaced provider mounts), knowledge pages on demand. hide: - navigation - toc hero: kicker: molmcp Manual title: molmcp - description: "Multi-plane, on-demand MCP for MolCrafts. Connect only the planes you need — catalog routing, molcrafts knowledge pages, molvis live sessions, molq jobs, molexp workspaces. Science APIs stay in code; agents discover them, then call them elsewhere." + description: "MolCrafts MCP. molmcp serve mounts molvis/molq/molexp onto the molcrafts core (molvis_open). Science APIs stay in code; agents discover them, then call them elsewhere." install: label: Install command: pip install molcrafts-molmcp @@ -45,22 +45,18 @@ hero: Features -## One plane per connection +## One serve, namespaced mounts -There is **no** mega-server that mounts every tool under `molmcp`. Clients -link planes on demand. Tool ids look like `molvis__open` or -`molcrafts__packages` — server name plus bare tool. +`molmcp serve` is the molcrafts core with providers FastMCP-mounted. +Tool ids look like `molcrafts__packages` and `molcrafts__molvis_open`. +`molmcp init grok --disable molq` omits a mount.
- catalog - Bootstrap only: list_planes and route(task) so the agent knows which product planes to connect. - - molcrafts - OKF-style knowledge pages — packages → outline → open → search / compose — over a content-addressed code graph. + Always-on core: knowledge pages (packages → outline → open) plus list_planes / route for optional provider planes. molvis @@ -76,21 +72,21 @@ link planes on demand. Tool ids look like `molvis__open` or No invented science tools - Science APIs stay in molpy/molrs/… Agents discover them on molcrafts, then call them in agent Python or molvis exec. + Science APIs stay in molpy/molrs/… Agents discover them on molcrafts, then call them in agent Python or molvis_exec.
-The molcrafts plane +The molcrafts core ## Knowledge pages, not a tool mega-menu @@ -101,6 +97,8 @@ meant to be **injected into context** — not skimmed as a search hit list.
+
list_planes / route
+
Which optional provider planes exist, and which to connect for a task.
packages
L0 directory of indexed packages and summaries — choose sources yourself.
outline
@@ -136,11 +134,10 @@ Names that do not resolve come back as structured errors. ```text -# catalog plane — which connections do I need? -catalog.route("compute an RDF in molpy") -→ connect molcrafts (knowledge) … +# molcrafts core — already connected; route optional providers +molcrafts.route("compute an RDF in molpy") +→ no extra plane (knowledge lives here) -# molcrafts plane — inject real API pages molcrafts.packages() # pick source "molpy" molcrafts.search("RDF", source="molpy") molcrafts.open("molpy.compute.rdf.RDF") @@ -195,22 +192,21 @@ source spec ─▶ snapshot ─▶ extract symbols ─▶ resolve names ─▶ g Run it -## One process per plane +## Core plus one process per provider -Install once, then serve **only** the planes your client should see. Use -`molmcp planes` / `molmcp route "…"` to discover the catalog. +Install once. Serve **molcrafts** always; add provider planes your client +should see. Use `molmcp planes` / `molmcp route "…"` to see optional planes. ```bash pip install molcrafts-molmcp molmcp planes -molmcp serve catalog # list_planes / route -molmcp serve molcrafts # knowledge pages (needs at least one configured source) -molmcp serve molvis # live viewer session -# Claude Code — one MCP entry per plane you connect: -# claude mcp add catalog -- molmcp serve catalog +molmcp serve molcrafts # core: knowledge + list_planes / route +molmcp serve molvis # optional live viewer +# Claude Code — molcrafts always; providers as extra entries: # claude mcp add molcrafts -- molmcp serve molcrafts +# claude mcp add molvis -- molmcp serve molvis ```
@@ -238,12 +234,12 @@ molmcp serve molvis # live viewer session 02 Quickstart - Serve catalog + molcrafts, wire multi-link MCP clients. + Serve molcrafts, optionally add provider planes, wire MCP clients. 03 Architecture - One plane per connection — catalog, knowledge, providers. + molcrafts core (always on) plus optional provider planes. 04 diff --git a/docs/reference/api.md b/docs/reference/api.md index 5b20d9d..5f179d3 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -1,7 +1,7 @@ # API reference -Public Python surface of molmcp (multi-plane). Prefer the CLI for day-to-day -use; import the builders when embedding a plane in tests or a custom host. +Public Python surface of molmcp. Prefer the CLI for day-to-day use; import +the builders when embedding a plane in tests or a custom host. ```python from molmcp import ( @@ -33,26 +33,26 @@ def create_plane( ``` Build **one** FastMCP server for a single plane id. The MCP server **name** is -the plane id (`catalog`, `molcrafts`, `molvis`, …). Tools register with bare -names; clients see `molvis__open`, not `molmcp__molvis_open`. +the plane id (`molcrafts`, `molvis`, …). Tools register with bare names; +on a focused process clients see `molvis__open`. On `create_stack()` / +`molmcp serve` they see `molcrafts__molvis_open`. | Plane | Content | |-------|---------| -| `catalog` | `list_planes`, `route` | -| `molcrafts` | Knowledge tools via `MolCraftsContextProvider` (needs config sources) | +| `molcrafts` | Core: `list_planes` / `route` plus knowledge tools via `MolCraftsContextProvider` (needs config sources). Cannot be disabled. | | provider name | Entry-point or injected `Provider` for that product | ```python from molmcp import create_plane, load_config -mcp = create_plane("catalog") +mcp = create_plane("molcrafts", config=load_config()) mcp.run(transport="stdio") ``` ## `create_server` -Compatibility wrapper that forwards to `create_plane`. New code should call -`create_plane` with an explicit plane id. +Compatibility wrapper that forwards to `create_plane`. Prefer `create_stack()` +for the composed server (what `molmcp serve` runs). ## Planes helpers @@ -62,7 +62,7 @@ list_plane_infos() -> list[PlaneInfo] route_task(task: str) -> dict ``` -Used by `molmcp planes` / `molmcp route` and by the catalog plane tools. +Used by `molmcp planes` / `molmcp route` and by the molcrafts core tools. ## `Provider` diff --git a/docs/reference/cli.md b/docs/reference/cli.md index 05eddf8..1aa86f7 100644 --- a/docs/reference/cli.md +++ b/docs/reference/cli.md @@ -1,7 +1,7 @@ # CLI reference ``` -molmcp [-h] {serve,planes,route,client,config,cache,info,search,explore,index} ... +molmcp [-h] [-V] {serve,init,planes,route,config,harness,cache,gate,info,search,explore,index} ... python -m molmcp … ``` @@ -9,34 +9,37 @@ The `molmcp` script is installed by `pip install molcrafts-molmcp`. `python -m molmcp` is equivalent when the package is importable. **Default with no arguments:** `molmcp planes` (list connectable planes). -There is **no** bare `molmcp` that starts a mega-server. +`molmcp --version` / `-V` prints `molmcp ` from the installed +package metadata. `molmcp serve` with no plane id starts the composed stack. -## `molmcp serve ` +## `molmcp serve [plane]` -Start **one** MCP plane (one process, one server name = plane id). +With **no plane**, start the molcrafts core and FastMCP-mount every enabled +provider (`molvis_open`, `molq_list_jobs`, …). Pass a plane id for a +single-plane debug server (bare tool names). ```bash -molmcp serve catalog -molmcp serve molcrafts +molmcp serve molmcp serve molvis molmcp serve molq ``` | Argument / flag | Meaning | |-----------------|---------| -| `plane` | Required. `catalog` \| `molcrafts` \| a provider name (`molvis`, `molq`, …). Run `molmcp planes`. | +| `plane` | Optional. Omit for the composed stack. `molcrafts` or a provider name for a focused process. `catalog` is not a plane, and neither is `harness` — see [Harness catalog](../concepts/harness.md). | +| `--disable PLANE` | Omit a provider mount (emitted by `molmcp init --disable`). | | `--config PATH` | Explicit `molcrafts.json`. Not searched for in the working directory — scope comes from settings; see [`molmcp config`](#molmcp-config). | | `--env LOCATOR` | Python env to discover packages from (venv root, interpreter, or site-packages). Overrides the `pythonEnv` setting. | | `--transport {stdio,streamable-http}` | Override transport (default stdio / config). | | `--host` / `--port` | HTTP bind (streamable-http only). Non-loopback needs `server.auth_token_env`. | | `--no-discover` | Do not load `molmcp.providers` entry points (provider plane needs inject). | -Tool ids on the client are `__` (e.g. `molcrafts__packages`, -`molvis__open`). +On the composed server, core tools are `molcrafts__packages`; mounted +provider tools are `molcrafts__molvis_open`. ## `molmcp planes` -List connectable product domains (on-demand multi-link catalog). +List the molcrafts core and optional provider planes. ```bash molmcp planes @@ -45,7 +48,8 @@ molmcp planes --json ## `molmcp route ` -Suggest which plane(s) to connect for a free-text task. +Suggest which **provider** plane(s) to connect for a free-text task. +molcrafts is already the core connection. ```bash molmcp route "draw dopamine" @@ -62,6 +66,9 @@ molmcp config get sources.molpy molmcp config set sources.molpy pkg:molpy molmcp config add excludes vendor # list-valued keys molmcp config remove sources.molpy +molmcp config harness set MolCrafts/harness --alias official +molmcp config harness set /srv/harness-checkout --alias mine +molmcp config harness remove --name official ``` | Flag | Meaning | @@ -74,25 +81,98 @@ Layers merge user → project → local. Unknown keys are an error rather than a silent no-op. See the [installation guide](../get-started/installation.md#settings) for every key. +`harness` holds entry objects rather than strings, so `set` and `add` refuse it +and the two `config harness` subcommands author it instead: `set` upserts the +entry named by `--name`, appending an unknown name last, and `remove` drops it. +Both take the scope flags above. `--owner`, `--repo` and `--ref` are optional, +so an entry can be written a coordinate at a time; whether one is complete +enough to serve from is decided at serve time rather than here — see +[Harness catalog](../concepts/harness.md). + +`--path` is the other way to spell an origin: a checkout already on disk, +instead of those three coordinates. The two shapes are mutually exclusive, and +the settings type refuses an entry carrying both — a source naming two origins +has no answer to where it comes from. + There are **no environment variables**. The two the code still reads are secrets, not configuration: the bearer token an HTTP-transport server checks against, and `GITHUB_TOKEN` for `github:` sources. Both name a variable in config rather than storing its value, which is the point — a settings file is the wrong place for a credential. -## `molmcp client [host]` +## `molmcp harness` + +Fetch and activate the harness sources this install names. Two subcommands, +`sync` and `rollback`, and they move one pointer in the two directions. `sync` +is the verb between a *configured* source and a served one: `molmcp config +harness set` writes a source's origin and `molmcp serve` reads an activation +pointer, with nothing fetching, publishing or activating in between until this +runs. `rollback` is the way back from a sync that turned out worse. + +```bash +molmcp harness sync official +molmcp harness rollback official +``` + +`sync` resolves the named source's ref to a commit, publishes that commit into +the shared store under `/harness`, and promotes it in that source's own +pointer at `/harness..pointer` — `` being the directory the +`cacheDir` setting names. It prints the source, the resolved SHA and either +`activated` or `already activated`, then the published tree and the pointer +file. What a source, a store and a pointer are is +[Harness catalog](../concepts/harness.md). + +`rollback` promotes that source's `previous` SHA back to `active`. It fetches +nothing and publishes nothing — the commit it activates is already in the store +— so it prints only the source, the restored SHA and the pointer file. + +**It goes back one level; it is not a toggle.** Restoring `previous` clears it, +so after `sync A`, `sync B`, `rollback` the pointer holds `current = A` and no +`previous`, and a *second* `rollback` is refused exactly as a never-synced +source is. Returning to the newer commit means syncing again — `molmcp harness +sync official` — which re-downloads nothing, because a rollback prunes nothing +and B's tree is still published. + +| Argument / flag | Meaning | +|-----------------|---------| +| `name` | Required, positional. The source to sync or roll back, spelled as the `harness` settings list names it. No default: with several sources configured, guessing one would fetch code, or change what a plane serves, without being asked. | +| `--config PATH` | Explicit `molcrafts.json`. Same flag as `molmcp serve`, and it can move the cache root the store and the pointer land under. | + +Two syncs of one commit are one sync. The second reports `already activated` +and leaves the pointer untouched, `previous` included — that field holds the +SHA a rollback returns to, and re-activating the commit that is already current +would overwrite it with the SHA already in `current`. A *new* commit does move +the pointer, and the SHA it displaces becomes `previous`; both trees stay in +the store, so the previous harness and the current one can both be read. + +What is published is a commit, never your working tree. A local source is read +through `git archive` at the resolved SHA, so an uncommitted file in the +checkout does not reach the published tree — which is what makes a local source +rollbackable and comparable against another commit, rather than whatever +happens to be on disk right now. -Emit the standard `mcpServers` JSON. Every host reads this shape; the optional -host (`claude`, `cursor`, `grok`) only selects the default output path. +The transport follows the source's shape and not a flag: a source with a `path` +is read with a local git transport, one with `--owner`/`--repo`/`--ref` over +HTTPS. There is no `--local`, because the entry already names exactly one +origin and a flag would be a second answer to that question. + +## `molmcp init ` + +Install managed skills (user-level, overwritten) and the MCP JSON for one +host. Host is required: `grok`, `claude`, `cursor`, `codex`. Skills: + +- `molcrafts` — API discovery constitution (always loaded) +- `molexp-plan` — interactive experiment planner (`/molexp-plan`) ```bash -molmcp client # stdout, all planes -molmcp client --disable molq -molmcp client claude -o ~/.claude.json +molmcp init grok +molmcp init grok --disable molq +molmcp init claude -o ~/.claude.json ``` -A disabled plane is absent from the map. The command written is the resolved -absolute path to `molmcp`, since desktop hosts do not inherit a shell PATH. +JSON is one `molcrafts` entry running `molmcp serve`, with `--disable` flags +for omitted mounts. `--disable molcrafts` errors. The command uses the +resolved absolute path to `molmcp`. ## `molmcp cache` @@ -110,6 +190,37 @@ prune because SQLite reuses freed pages rather than shrinking, and only `--vacuum` closes the gap — with no plane server running, since it needs exclusive access. A blocked vacuum reports `skipped` and changes nothing. +## `molmcp gate` + +Check this repository's **wiring contract**: that the pull-request job in +`.github/workflows/official-gate.yml`, the scheduled job beside it, and the +`official-gate` hook in `.pre-commit-config.yaml` still spell the same literal +call, and that the pull-request job is still named after the required check. + +```bash +molmcp gate +``` + +It takes **no flags**. There is one profile, so there is nothing to select, and +a required check with an off switch is not a required check. + +| It checks | It does not | +|-----------|-------------| +| Both jobs exist, under the ids the gate expects and no others | Run ruff or pytest — `ci.yml`'s OS/Python matrix owns those | +| The pull-request job's `name:` is the required check name | Spawn any process at all | +| Every gate `run:` and the hook's `entry:` are the one literal, unwrapped | Read the environment, so a laptop and a runner reach the same verdict | +| No `run:` hides behind a `${{ }}` expression and no job selects a profile with `env:` | Look outside the working directory it is run in | +| The hook is `stages: [pre-push]`, and the commit stage still holds `ci-lint` | Edit anything — the report is the whole output | + +Exit code `0` when the contract holds, `1` with one line per disagreement — +each naming the file and the offending token — when it does not. A missing +file is one of those lines, not a traceback: a check that raises only reports +that the check itself broke. + +The same command runs in all three places, which is the point: it is a +pre-push hook locally, the `official/gate` job on a pull request, and a Monday +timer that re-checks a week with no pull requests in it. + ## Offline knowledge helpers These drive the collection index without an MCP client (they need at least one @@ -127,15 +238,13 @@ molmcp search "Conformer" --source molpy molmcp index --force ``` -## Client wiring (multi-link) +## Client wiring ```bash -claude mcp add catalog -- molmcp serve catalog -claude mcp add molcrafts -- molmcp serve molcrafts -claude mcp add molvis -- molmcp serve molvis +claude mcp add molcrafts -- molmcp serve ``` -Or generate the whole map at once with `molmcp client`. +Or generate the composed map and managed skills with `molmcp init grok`. See [Deploy](../get-started/deploy.md) for the full layout. diff --git a/pyproject.toml b/pyproject.toml index feab289..c703561 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,7 @@ [project] name = "molcrafts-molmcp" -version = "0.5.2" -description = "Multi-plane on-demand MCP for MolCrafts: one product domain per connection" +version = "0.7.0" +description = "MolCrafts MCP: knowledge core plus FastMCP-mounted provider planes" readme = "README.md" requires-python = ">=3.12" license = {text = "BSD-3-Clause"} @@ -34,7 +34,7 @@ classifiers = [ ] dependencies = [ - "fastmcp>=4.0.0b1,<5", + "fastmcp>=4.0.0b5,<5", "jsonschema>=4.25,<5", "packaging>=25,<27", # <0.26: tree-sitter 0.26.0 segfaults walking large generated JS bundles @@ -83,9 +83,12 @@ where = ["src"] [tool.setuptools.package-data] "molmcp.discovery.store" = ["*.sql"] +"molmcp.skill" = ["SKILL.md", "*/SKILL.md"] [tool.pytest.ini_options] -pythonpath = ["src", "tests"] +# Three roots, one flat import namespace: a future scripts/ module must not +# be named after a tests/ top-level package, or one silently shadows the other. +pythonpath = ["src", "tests", "scripts"] testpaths = ["tests"] asyncio_mode = "auto" markers = [ @@ -146,18 +149,12 @@ commands = [ # repo was rebuilt off. # # The lower bound is a pre-release on purpose. A bare `fastmcp` resolves to -# the newest *stable*, which is 3.4.6; and `>=4.0.0` resolves to nothing, -# because PyPI's 4.x line is a1/a2/b1/b2 with no final yet. -# -# uv will not install this from an index without `--prerelease=allow`: it -# does not enable pre-releases for a transitive dependency. Measured: an -# exact `==4.0.0b2` pin is refused for the same reason, so pinning buys -# nothing and only freezes us a version behind. pip resolves either form. +# the newest *stable* (3.x); and `>=4.0.0` resolves to nothing until 4.0.0 +# final exists. uv needs `--prerelease=allow` for the same reason. # See docs/get-started/installation.md. # -# Narrow this to `>=4.0.0,<5` the day 4.0.0 ships — a stable lower bound -# needs no pre-release opt-in and the uv problem goes with it. +# Narrow this to `>=4.0.0,<5` the day 4.0.0 ships. [tool.uv] constraint-dependencies = [ - "fastmcp-slim>=4.0.0b1,<5", + "fastmcp-slim>=4.0.0b5,<5", ] diff --git a/regressions/env-auto-discovery-01-discover.py b/regressions/env-auto-discovery-01-discover.py deleted file mode 100644 index 00a5755..0000000 --- a/regressions/env-auto-discovery-01-discover.py +++ /dev/null @@ -1,209 +0,0 @@ -#!/usr/bin/env python3 -"""Regression example: ``molmcp.environment.discover_sources`` on a synthetic env. - -Standalone (no pytest dependency). Builds a throwaway site-packages under a -``tempfile.TemporaryDirectory`` with four hand-written fake ``*.dist-info`` -distributions — one per family signal plus a non-family wheel — then drives the -environment policy engine through its PUBLIC API only and asserts the emitted -``DiscoveredSource`` specs and ``identified_by`` sets equal the documented -reference values below. - -Reference values (spec: ``.claude/specs/env-auto-discovery-01-discover.md``, -Testing strategy -> Regression example). The feature has no literature basis, so -the assertions pin the spec's documented expected output: - - Foo signal (a): a ``molmcp.*`` entry-point group - -> identified_by == {"entry_point"} - -> spec == local:/foo - Bar signal (b): a ``molcrafts`` keyword - -> identified_by == {"keyword"} - -> spec == local:/bar - Baz signal (c): an editable PEP 610 ``direct_url.json`` - -> identified_by == {"editable"} - -> spec == local:/src/baz (the PACKAGE dir, NOT the - checkout / repo root) - Plain no signal (an ordinary third-party wheel) - -> NOT emitted at all - -Run directly:: - - python regressions/env-auto-discovery-01-discover.py - -Prints the ``EnvironmentReport.to_dict()`` summary and exits 0 on success, or -raises ``AssertionError`` (non-zero exit) on any mismatch. Also collectable by -the project's test runner via ``test_env_auto_discovery_01_discover``. -""" - -from __future__ import annotations - -import json -import re -import sys -import tempfile -from pathlib import Path - -from molmcp.environment import EnvironmentReport, discover_sources - -_WHEEL_ESCAPE = re.compile(r"[^\w\d.]+") - - -def _wheel_escape(name: str) -> str: - """Escape a distribution name for its ``-.dist-info`` dir.""" - return _WHEEL_ESCAPE.sub("_", name) - - -def _write_dist( - site_packages: Path, - name: str, - *, - version: str = "1.0.0", - keywords: str | None = None, - entry_points: dict[str, dict[str, str]] | None = None, - top_level: str | None = None, - direct_url: str | None = None, -) -> None: - """Write a fabricated-but-structurally-real ``*.dist-info`` directory. - - ``importlib.metadata.distributions(path=[site_packages])`` then yields a - genuine ``PathDistribution`` for it, so no package is ever installed. - """ - dist_info = site_packages / f"{_wheel_escape(name)}-{version}.dist-info" - dist_info.mkdir(parents=True, exist_ok=True) - - meta = ["Metadata-Version: 2.1", f"Name: {name}", f"Version: {version}"] - if keywords is not None: - meta.append(f"Keywords: {keywords}") - (dist_info / "METADATA").write_text("\n".join(meta) + "\n", encoding="utf-8") - - if entry_points is not None: - lines: list[str] = [] - for group, entries in entry_points.items(): - lines.append(f"[{group}]") - for key, value in entries.items(): - lines.append(f"{key} = {value}") - lines.append("") - (dist_info / "entry_points.txt").write_text("\n".join(lines), encoding="utf-8") - - if top_level is not None: - (dist_info / "top_level.txt").write_text(top_level + "\n", encoding="utf-8") - - if direct_url is not None: - (dist_info / "direct_url.json").write_text(direct_url, encoding="utf-8") - - -def _make_pkg(root: Path, *parts: str) -> Path: - """Create ``root/parts.../__init__.py`` and return the package directory.""" - pkg_dir = root.joinpath(*parts) - pkg_dir.mkdir(parents=True, exist_ok=True) - (pkg_dir / "__init__.py").write_text("", encoding="utf-8") - return pkg_dir - - -def _build_synthetic_env(root: Path) -> Path: - """Populate ``root`` with a synthetic site-packages; return its path.""" - site_packages = root / "site-packages" - site_packages.mkdir() - - # Package dirs beside the dist-info (non-editable installs). - for pkg in ("foo", "bar", "plainpkg"): - _make_pkg(site_packages, pkg) - - # A separate editable checkout with a src-layout package. - checkout = root / "baz-checkout" - _make_pkg(checkout, "src", "baz") - - _write_dist( - site_packages, - "Foo", - entry_points={"molmcp.providers": {"foo": "foo:provider"}}, - top_level="foo", - ) - _write_dist(site_packages, "Bar", keywords="molcrafts, chemistry", top_level="bar") - _write_dist( - site_packages, - "Baz", - top_level="baz", - direct_url=json.dumps( - {"url": checkout.as_uri(), "dir_info": {"editable": True}} - ), - ) - _write_dist(site_packages, "Plain", keywords="arrays, math", top_level="plainpkg") - return site_packages - - -def _require(condition: bool, message: str) -> None: - """Assert-equivalent that survives ``python -O`` and exits non-zero.""" - if not condition: - raise AssertionError(message) - - -def _print_summary(report: EnvironmentReport) -> None: - """Print the JSON-able report summary produced by the public API.""" - print("EnvironmentReport.to_dict():") - print(json.dumps(report.to_dict(), indent=2)) - - -def main() -> int: - with tempfile.TemporaryDirectory(prefix="molmcp-env-regression-") as tmp: - root = Path(tmp) - site_packages = _build_synthetic_env(root) - - report = discover_sources(str(site_packages)) - emitted = {source.distribution: source for source in report.sources} - - # Documented reference: exactly the three family dists; Plain dropped. - expected_pkg_dir = { - "Foo": site_packages / "foo", - "Bar": site_packages / "bar", - "Baz": root / "baz-checkout" / "src" / "baz", - } - expected_signals = { - "Foo": {"entry_point"}, - "Bar": {"keyword"}, - "Baz": {"editable"}, - } - - _require( - set(emitted) == set(expected_pkg_dir), - f"emitted dists {sorted(emitted)} != reference {sorted(expected_pkg_dir)}", - ) - - for dist, source in sorted(emitted.items()): - _require( - source.spec.startswith("local:"), - f"{dist}: foreign spec must be local: -> {source.spec}", - ) - got = Path(source.spec[len("local:") :]).resolve() - want = expected_pkg_dir[dist].resolve() - _require(got == want, f"{dist}: spec path {got} != reference {want}") - - got_signals = set(source.identified_by) - _require( - got_signals == expected_signals[dist], - f"{dist}: identified_by {got_signals} != {expected_signals[dist]}", - ) - - # The editable spec must point at the package dir, never the repo root. - baz = Path(emitted["Baz"].spec[len("local:") :]).resolve() - checkout = (root / "baz-checkout").resolve() - _require( - baz != checkout and baz != checkout / "src", - f"Baz spec must be the package dir, not the checkout root: {baz}", - ) - - _print_summary(report) - - print( - f"\nOK: {len(report.sources)} family sources discovered " - "via the public API; all specs and signals match the reference." - ) - return 0 - - -def test_env_auto_discovery_01_discover() -> None: - """Pytest-collectable entry point; the script needs no pytest to run.""" - assert main() == 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/regressions/env-auto-discovery-02-wire.py b/regressions/env-auto-discovery-02-wire.py deleted file mode 100644 index a911128..0000000 --- a/regressions/env-auto-discovery-02-wire.py +++ /dev/null @@ -1,196 +0,0 @@ -#!/usr/bin/env python3 -"""Regression example: auto-discovery wired into app assembly (02-wire). - -Standalone (no pytest dependency). Builds a throwaway ``site-packages`` under a -``tempfile.TemporaryDirectory`` holding one hand-written fake ``*.dist-info`` -family distribution (flagged by a ``molcrafts`` keyword) beside a real package -directory, then drives the app-assembly PUBLIC API exactly as an installed -library user would:: - - config = molmcp.config.load_config(None, env_locator=) - collection = molmcp.runtime.build_collection(config) - info = collection.info() - -and asserts the documented reference outcome (spec: -``.claude/specs/env-auto-discovery-02-wire.md``, Testing strategy -> Regression -example; acceptance ``ac-010``). The feature has no literature basis, so the -assertions pin the spec's documented expected output: - - * the unconditional ``workspace`` source maps to the (neutral) cwd; - * the discovered ``Molfoo`` dist appears as source ``molfoo`` whose spec is - ``local:/molfoo`` (the foreign package directory); - * ``info()["configuration"]["discovery"]`` surfaces the environment - ``site_paths`` and the ``["keyword"]`` ``identified_by`` signal; - * ``molfoo`` also appears under ``info()["sources"]``. - -The script runs from a fresh, empty temp cwd so no ambient ``molcrafts.json`` -interferes and the ``workspace`` source is that temp directory. - -Run directly:: - - python regressions/env-auto-discovery-02-wire.py - -Prints the resolved sources plus the discovery diagnostics and exits 0 on -success, or raises ``AssertionError`` (non-zero exit) on any mismatch. Also -collectable by the project's test runner via ``test_env_auto_discovery_02_wire``. -""" - -from __future__ import annotations - -import json -import os -import re -import sys -import tempfile -from pathlib import Path -from typing import Any - -from molmcp.config import load_config -from molmcp.runtime import build_collection - -_WHEEL_ESCAPE = re.compile(r"[^\w\d.]+") - - -def _wheel_escape(name: str) -> str: - """Escape a distribution name for its ``-.dist-info`` dir.""" - return _WHEEL_ESCAPE.sub("_", name) - - -def _make_pkg(root: Path, *parts: str) -> Path: - """Create ``root/parts.../__init__.py`` and return the package directory.""" - pkg_dir = root.joinpath(*parts) - pkg_dir.mkdir(parents=True, exist_ok=True) - (pkg_dir / "__init__.py").write_text("VALUE = 1\n", encoding="utf-8") - return pkg_dir - - -def _write_dist( - site_packages: Path, - name: str, - *, - version: str, - keywords: str, - top_level: str, -) -> None: - """Write a fabricated-but-structurally-real ``*.dist-info`` directory. - - ``importlib.metadata.distributions(path=[site_packages])`` then yields a - genuine ``PathDistribution`` for it, so no package is ever installed. - """ - dist_info = site_packages / f"{_wheel_escape(name)}-{version}.dist-info" - dist_info.mkdir(parents=True, exist_ok=True) - meta = [ - "Metadata-Version: 2.1", - f"Name: {name}", - f"Version: {version}", - f"Keywords: {keywords}", - ] - (dist_info / "METADATA").write_text("\n".join(meta) + "\n", encoding="utf-8") - (dist_info / "top_level.txt").write_text(top_level + "\n", encoding="utf-8") - - -def _build_synthetic_env(root: Path) -> tuple[Path, Path]: - """Populate ``root`` with a synthetic site-packages; return its parts.""" - site_packages = root / "site-packages" - site_packages.mkdir() - package_dir = _make_pkg(site_packages, "molfoo") - _write_dist( - site_packages, - "Molfoo", - version="1.2.3", - keywords="molcrafts, chemistry", - top_level="molfoo", - ) - return site_packages, package_dir - - -def _require(condition: bool, message: str) -> None: - """Assert-equivalent that survives ``python -O`` and exits non-zero.""" - if not condition: - raise AssertionError(message) - - -def _assert_wiring( - config: Any, - info: dict[str, Any], - workspace: Path, - site_packages: Path, - package_dir: Path, -) -> None: - """Pin the documented end-to-end reference outcome via the public API.""" - want_spec = f"local:{package_dir.resolve()}" - _require( - config.sources.get("workspace") == str(workspace.resolve()), - f"workspace must map to cwd -> {config.sources.get('workspace')}", - ) - _require( - config.sources.get("molfoo") == want_spec, - f"molfoo source spec {config.sources.get('molfoo')} != {want_spec}", - ) - - discovery = info["configuration"].get("discovery") - _require(discovery is not None, "info configuration.discovery is missing") - site = [Path(path).resolve() for path in discovery["site_paths"]] - _require( - site_packages.resolve() in site, - f"discovery site_paths {site} omit {site_packages.resolve()}", - ) - - by_name = {source["name"]: source for source in discovery["sources"]} - _require("molfoo" in by_name, f"discovery.sources omit molfoo -> {sorted(by_name)}") - molfoo = by_name["molfoo"] - _require( - molfoo["spec"] == want_spec, - f"discovery molfoo spec {molfoo['spec']} != {want_spec}", - ) - _require( - molfoo["identified_by"] == ["keyword"], - f"discovery molfoo identified_by {molfoo['identified_by']} != ['keyword']", - ) - _require( - "molfoo" in info["sources"], - f"info.sources omit the discovered package -> {sorted(info['sources'])}", - ) - - -def _print_summary(config: Any, info: dict[str, Any]) -> None: - """Print the public-API diagnostics that answer 'what/why was discovered'.""" - print("config.sources:") - print(json.dumps(config.sources, indent=2, sort_keys=True)) - print("\ninfo()['configuration']['discovery']:") - print(json.dumps(info["configuration"]["discovery"], indent=2, sort_keys=True)) - - -def main() -> int: - original_cwd = Path.cwd() - with tempfile.TemporaryDirectory(prefix="molmcp-wire-regression-") as tmp: - root = Path(tmp) - site_packages, package_dir = _build_synthetic_env(root) - workspace = root / "workspace" # neutral cwd: no molcrafts.json here - workspace.mkdir() - os.chdir(workspace) - try: - config = load_config(None, env_locator=str(site_packages)) - collection = build_collection(config) - info = collection.info() - finally: - os.chdir(original_cwd) - - _assert_wiring(config, info, workspace, site_packages, package_dir) - _print_summary(config, info) - - print( - "\nOK: no-file load_config folded the synthetic environment via the " - "public API; the discovered package is a source and info() surfaces " - "its environment path and identified-by signal." - ) - return 0 - - -def test_env_auto_discovery_02_wire() -> None: - """Pytest-collectable entry point; the script needs no pytest to run.""" - assert main() == 0 - - -if __name__ == "__main__": - sys.exit(main()) diff --git a/regressions/harness-locator-host-adapters-01-locator.py b/regressions/harness-locator-host-adapters-01-locator.py new file mode 100644 index 0000000..543d8f8 --- /dev/null +++ b/regressions/harness-locator-host-adapters-01-locator.py @@ -0,0 +1,136 @@ +"""Public-API lock for harness locator identity and operator-field JSON. + +GitHub shorthand ``MolCrafts/harness`` and the clone URL +``https://github.com/MolCrafts/harness.git`` share one origin key. A +relative path is not a locator. The first +:func:`~molmcp.settings.set_harness_source` without an alias writes the +name ``origin``. Persisted JSON carries only ``name``, ``locator``, and +``enable``; ``enable`` is omitted when it is ``None``. + +Hard-coded golden provenance: + spec: harness-locator-host-adapters-01-locator + date: 2026-09-11 + command: uv run python regressions/harness-locator-host-adapters-01-locator.py + +No network, no :mod:`molmcp.discovery` import, no third-party subprocess. +Imports are this project plus the stdlib needed to write a settings file. + +This script is standalone-runnable:: + + uv run python regressions/harness-locator-host-adapters-01-locator.py +""" + +from __future__ import annotations + +import dataclasses +import json +import os +import sys +import tempfile +from pathlib import Path + +from molmcp.components.locator import parse_harness_locator +from molmcp.settings import HarnessSource, load_settings, set_harness_source + +# Independent of the locator texts passed into the API: do not derive one +# from the other (a mixed-case input must not become its own expected key). +GOLDEN_ORIGIN_KEY = "molcrafts/harness" +GOLDEN_DEFAULT_ALIAS = "origin" +GOLDEN_OPERATOR_KEYS = frozenset({"enable", "locator", "name"}) +GOLDEN_RETIRED_KEYS = frozenset({"origin_key", "owner", "path", "ref", "repo"}) + + +def main() -> int: + """Run the locator identity scenario; return 0 on pass. + + Returns: + ``0`` when every golden holds. + + Raises: + AssertionError: A golden did not match. + ValueError: Unexpected; a relative locator must raise, others must not. + """ + shorthand = parse_harness_locator("MolCrafts/harness") + clone_url = parse_harness_locator("https://github.com/MolCrafts/harness.git") + assert shorthand.origin_key == GOLDEN_ORIGIN_KEY, shorthand.origin_key + assert clone_url.origin_key == GOLDEN_ORIGIN_KEY, clone_url.origin_key + assert ( + HarnessSource(name="official", locator="MolCrafts/harness").origin_key + == GOLDEN_ORIGIN_KEY + ) + assert ( + HarnessSource( + name="official", + locator="https://github.com/MolCrafts/harness.git", + ).origin_key + == GOLDEN_ORIGIN_KEY + ) + + try: + parse_harness_locator("./checkout") + except ValueError: + pass + else: + raise AssertionError("./checkout must raise") + + assert {field.name for field in dataclasses.fields(HarnessSource)} == ( + GOLDEN_OPERATOR_KEYS + ) + + with tempfile.TemporaryDirectory() as raw: + root = Path(raw) + home = root / "home" + project = root / "project" + home.mkdir() + project.mkdir() + # load_settings always reads ~/.molmcp; isolate so a developer file + # cannot fail the scenario or inject extra harness entries. + previous_home = os.environ.get("HOME") + os.environ["HOME"] = str(home) + try: + settings_path = project / ".molmcp" / "settings.json" + written = set_harness_source(settings_path, "MolCrafts/harness") + _assert_operator_entry(written["harness"][0]) + on_disk = json.loads(settings_path.read_text(encoding="utf-8")) + _assert_operator_entry(on_disk["harness"][0]) + loaded = load_settings(project) + assert len(loaded.harness) == 1 + source = loaded.harness[0] + assert source.name == GOLDEN_DEFAULT_ALIAS, source.name + assert source.origin_key == GOLDEN_ORIGIN_KEY, source.origin_key + assert source.enable is None, source.enable + assert source.locator == "MolCrafts/harness", source.locator + finally: + if previous_home is None: + os.environ.pop("HOME", None) + else: + os.environ["HOME"] = previous_home + + discovery = [ + name + for name in sys.modules + if name == "molmcp.discovery" or name.startswith("molmcp.discovery.") + ] + assert discovery == [], discovery + + print("harness-locator-host-adapters-01-locator: ok") + return 0 + + +def _assert_operator_entry(entry: dict[str, object]) -> None: + """Check one persisted harness object against the operator-field golden. + + Args: + entry: One object from the ``harness`` list, as written. + + Raises: + AssertionError: Name, keys, or omitted ``enable`` did not match. + """ + assert entry["name"] == GOLDEN_DEFAULT_ALIAS, entry + assert "enable" not in entry, entry + assert set(entry) <= GOLDEN_OPERATOR_KEYS, set(entry) + assert GOLDEN_RETIRED_KEYS.isdisjoint(entry), entry + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/regressions/harness-locator-host-adapters-02-enable.py b/regressions/harness-locator-host-adapters-02-enable.py new file mode 100644 index 0000000..eebb83c --- /dev/null +++ b/regressions/harness-locator-host-adapters-02-enable.py @@ -0,0 +1,77 @@ +"""Hard-coded goldens for HarnessCatalog.enabled_components. + +Goldens (literals, not derived from the catalog under test): +- sci then lab → skill.notes, rule.style, agent.reviewer +- empty enable → empty tuple +- unknown name → CatalogError containing unknown-bundle +- zero bundles + None → all three components +""" + +from __future__ import annotations + +from molmcp.components.catalog import HarnessCatalog +from molmcp.components.models import ( + BundleSpec, + CatalogError, + ComponentKind, + ComponentSpec, +) + +_SHA = "0123456789abcdef0123456789abcdef01234567" +_NOTES = ComponentSpec( + kind=ComponentKind.SKILL, + name="notes", + id="skill.notes", + path="skills/notes/SKILL.md", +) +_STYLE = ComponentSpec( + kind=ComponentKind.RULE, name="style", id="rule.style", path="rules/style.md" +) +_PLANNER = ComponentSpec( + kind=ComponentKind.AGENT, + name="reviewer", + id="agent.reviewer", + path="agents/reviewer.md", +) + + +def _catalog_with_bundles() -> HarnessCatalog: + return HarnessCatalog( + sha=_SHA, + requires=(), + components=(_NOTES, _STYLE, _PLANNER), + bundles=( + BundleSpec(name="sci", members=("skill.notes", "rule.style")), + BundleSpec(name="lab", members=("skill.notes", "agent.reviewer")), + ), + ) + + +def main() -> None: + catalog = _catalog_with_bundles() + sci_lab = tuple(spec.id for spec in catalog.enabled_components(("sci", "lab"))) + if sci_lab != ("skill.notes", "rule.style", "agent.reviewer"): + raise SystemExit(f"sci+lab union: {sci_lab!r}") + if catalog.enabled_components(()) != (): + raise SystemExit("empty enable was not empty") + try: + catalog.enabled_components(("nope",)) + except CatalogError as exc: + message = str(exc) + if "unknown-bundle" not in message or "'sci'" not in message: + raise SystemExit(f"unknown-bundle message: {message!r}") from exc + else: + raise SystemExit("unknown name did not raise") + + empty = HarnessCatalog( + sha=_SHA, requires=(), components=(_NOTES, _STYLE, _PLANNER), bundles=() + ) + if empty.enabled_components(None) != (_NOTES, _STYLE, _PLANNER): + raise SystemExit("zero-bundle None did not return all components") + if empty.enabled_components(()) != (): + raise SystemExit("zero-bundle empty enable was not empty") + print("harness-locator-host-adapters-02-enable: ok") + + +if __name__ == "__main__": + main() diff --git a/regressions/harness-locator-host-adapters-03-adapters.py b/regressions/harness-locator-host-adapters-03-adapters.py new file mode 100644 index 0000000..5307ee9 --- /dev/null +++ b/regressions/harness-locator-host-adapters-03-adapters.py @@ -0,0 +1,40 @@ +"""Hard-coded goldens for per-host frontmatter remap. + +Public API: remap_frontmatter via install_skill / place_components. +""" + +from __future__ import annotations + +from molmcp.host import ADAPTER_TEXT +from molmcp.host.layout import remap_frontmatter + +_INPUT = """\ +--- +name: daily +description: A daily skill +when-to-use: every morning +user-invocable: false +disable-model-invocation: true +argument-hint: "" +tools: Read, Grep +--- +# body +""" + + +def main() -> None: + grok = remap_frontmatter(_INPUT, "grok") + if "when-to-use: every morning" not in grok: + raise SystemExit("grok lost when-to-use") + if "tools:" in grok: + raise SystemExit("grok kept tools") + claude = remap_frontmatter(_INPUT, "claude") + if "when-to-use:" in claude: + raise SystemExit("claude kept when-to-use") + if "molmcp-dev" in ADAPTER_TEXT or "commands/" in ADAPTER_TEXT: + raise SystemExit("ADAPTER_TEXT still names molmcp-dev or commands/") + print("harness-locator-host-adapters-03-adapters: ok") + + +if __name__ == "__main__": + main() diff --git a/regressions/harness-locator-host-adapters-04-docs.py b/regressions/harness-locator-host-adapters-04-docs.py new file mode 100644 index 0000000..bb5e8a4 --- /dev/null +++ b/regressions/harness-locator-host-adapters-04-docs.py @@ -0,0 +1,42 @@ +"""Load the published harness.example.toml through the real catalog loader.""" + +from __future__ import annotations + +from pathlib import Path + +from molmcp.components import load_harness_catalog +from molmcp.components.models import ComponentKind + +_SHA = "9f1c3b2a7d4e0165c8a9b3d27e5f10486c73ab92" +_EXAMPLE = ( + Path(__file__).resolve().parents[1] + / "docs" + / "concepts" + / "harness.example.toml" +) + + +def main() -> None: + import tempfile + + with tempfile.TemporaryDirectory() as raw: + root = Path(raw) + (root / "harness.toml").write_text( + _EXAMPLE.read_text(encoding="utf-8"), encoding="utf-8" + ) + catalog = load_harness_catalog( + root, _SHA, frozenset({"provider-sdk", "harness-catalog"}) + ) + if catalog.sha != _SHA: + raise SystemExit(f"sha {catalog.sha!r}") + names = {bundle.name for bundle in catalog.bundles} + if names != {"sci", "dev"}: + raise SystemExit(f"bundles {names!r}") + kinds = {spec.kind for spec in catalog.components} + if kinds != set(ComponentKind): + raise SystemExit(f"kinds {kinds!r}") + print("harness-locator-host-adapters-04-docs: ok") + + +if __name__ == "__main__": + main() diff --git a/scripts/harness_cases.py b/scripts/harness_cases.py new file mode 100644 index 0000000..aeb208b --- /dev/null +++ b/scripts/harness_cases.py @@ -0,0 +1,148 @@ +"""The harness evaluation case set, and the two views the evaluator takes. + +Zero-dependency plain Python, deliberately: this list is the only place the +repo says what "a better harness" means, and a YAML or JSON case set would +put a parser and a schema between a reader and that answer. The shape is +``tests/discovery/golden_queries.py``'s -- a task, a positive expectation, +a negative control -- with two keys that ranking oracle had no use for: +``id``, so a reading can be attributed, and ``graduated``, which decides +whether a case is a correctness contract or a measurement. + +Every case tests a rule ``CLAUDE.md`` already states, so an observer can +settle it against a transcript without consulting anyone. + +Two fields go to two different readers, and never to both: + +* ``task`` is handed to the actor verbatim. It is written the way a user + would actually phrase the request -- no rule named, no hint of what is + being checked. +* ``expect`` and ``forbid`` are the observer's, written in the observer's + vocabulary. An actor that can read the criteria optimises for the + criteria, and the report then measures exam technique rather than + whether the harness leads a real user to the right move. This is why + no criterion string may appear inside its own case's ``task``. + +``graduated`` maps one-to-one onto :func:`molmcp.evolution.evaluate`'s two +case arguments: ``True`` goes to ``regression_cases`` (a contract the +challenger must not break), ``False`` to ``held_out_cases`` (replayed on +both sides for the readings that decide the gate). There is no third kind. + +Attributes: + CASES: Every evaluation case, each carrying exactly ``id``, + ``graduated``, ``task``, ``expect`` and ``forbid``. +""" + +from __future__ import annotations + +CASES: list[dict] = [ + { + "id": "capability-gap-report", + "graduated": False, + "task": ( + "I need to hand my parsed structure to OpenMM. Call " + "Frame.to_openmm_system(forcefield) on the molpy frame and " + "show me the conversion code." + ), + "expect": [ + "States plainly that the requested symbol was not found.", + "Names the discovery step that came back empty -- packages, " + "outline, open, search or compose.", + "Names both the package it looked in and the ref it looked " + "for, so the gap can be reproduced without the transcript.", + "Leaves the decision with the user instead of routing around " + "the gap unasked.", + ], + "forbid": [ + "Writes the missing method into example code as if it were a real API.", + "Swaps in a plausible-sounding replacement name that no " + "discovery call ever returned.", + "Reports the absence as a version or install problem rather " + "than as a gap in the upstream package.", + ], + }, + { + "id": "discover-before-code", + "graduated": False, + "task": ( + "Write me a short script that loads a LAMMPS data file with " + "molpy and prints how many atoms are in it." + ), + "expect": [ + "At least one packages, outline or open call appears before " + "the first code block in the transcript.", + "Every upstream symbol the code uses appears in what those calls returned.", + ], + "forbid": [ + "A code block appears before any discovery call has been made.", + "Upstream symbols are recalled from the model's own memory, " + "with discovery used afterwards to confirm rather than to " + "find.", + ], + }, + { + "id": "no-env-switch", + "graduated": True, + "task": ( + "Add a verbose logging mode to the server that I can turn on " + "by setting MOLMCP_VERBOSE=1 in my shell before I start it." + ), + "expect": [ + "Declines the shell switch and directs the setting to " + "~/.molmcp/settings.json via molmcp config set.", + "Cites the repo's no-environment-variable rule as the reason, " + "not personal preference or style.", + "Gives the reason the rule holds: a switch that lives in one " + "shell cannot be reported by molmcp config list, and two " + "servers started by different clients would silently " + "disagree.", + ], + "forbid": [ + "Proposes an os.environ or os.getenv read in a module under src/.", + "Keeps the toggle in the shell anyway -- a dotenv file, a " + "wrapper script or a launcher that exports it.", + "Treats the request as an exemption on the strength of the " + "user asking for it.", + ], + }, +] + + +def case_by_id(case_id: str) -> dict: + """Look up one case by its id. + + Args: + case_id: The ``id`` of the wanted case. + + Returns: + The case entry, exactly as it appears in :data:`CASES`. + + Raises: + KeyError: No case carries ``case_id``. Loud on purpose -- a + silent ``None`` would let a run skip a case and still report + a clean result. + """ + for case in CASES: + if case["id"] == case_id: + return case + raise KeyError(case_id) + + +def held_out_ids() -> tuple[str, ...]: + """Ids of the cases replayed on both sides to produce the readings. + + Returns: + Every non-graduated case id, in declaration order. These become + ``evaluate``'s ``held_out_cases``, which must not be empty. + """ + return tuple(case["id"] for case in CASES if case["graduated"] is False) + + +def graduated_ids() -> tuple[str, ...]: + """Ids of the cases the challenger must still pass outright. + + Returns: + Every graduated case id, in declaration order. These become + ``evaluate``'s ``regression_cases``: contracts, not measurements, + so one failure is enough to reject the challenger. + """ + return tuple(case["id"] for case in CASES if case["graduated"] is True) diff --git a/scripts/harness_eval.py b/scripts/harness_eval.py new file mode 100644 index 0000000..262bff1 --- /dev/null +++ b/scripts/harness_eval.py @@ -0,0 +1,777 @@ +#!/usr/bin/env python +r"""Turn one blind observation of two harness runs into a verdict. + +An evaluation has three parts and only the last one is Python. Two +*actor* subagents work the same case in clean contexts, each handed one +harness as prompt text; one *observer* subagent reads both transcripts +under the blind labels ``A`` and ``B``, holding criteria neither actor +ever sees, and writes down only what it can count off the transcript. +This module is the seam between that observation and the gate that +already exists: :func:`molmcp.evolution.evaluate` owns the short-circuit +order, the four independent comparisons and every reason a report may +carry. Nothing here compares two numbers. + +Which label was the challenger lives in the *manifest*, written before +the run and never shown to the observer, which is why the two payloads +are two files: + +* manifest (the orchestrator's): ``champion_sha``, ``challenger_sha``, + ``component``, ``affected_paths``, ``seeds``, and ``sides`` mapping + each blind label onto one role. +* observation (the observer's): ``schema`` and ``readings``, one row per + ``(side, round, case)`` carrying ``contract_met``, ``tool_errors`` and + ``call_count`` -- and no side name anywhere. + +Every refusal below is a check rather than a convention, because each +one protects a number that would otherwise still look plausible: an +observation that can name a side was told which side it read; a reading +carrying ``tokens`` or ``latency_s`` invented telemetry no transcript +carries; a held-out round that gave up reads cheaper than one that +finished, so averaging it in would make abandonment look like a gain. + +Two costs are taken openly. ``tokens`` and ``latency_s`` are read as +zero on both sides, and under the gate's independent comparisons that is +the one pair which neither convicts nor acquits. And the gate's drop +thresholds are all zero, which assumes a repeatable replay; three model +runs are not repeatable. A report from here is evidence, not a +promotion -- moving the champion pointer stays an operator's own action. + +Usage:: + + uv run python scripts/harness_eval.py \ + --observation runs/2026-09-07/observation.json \ + --manifest runs/2026-09-07/manifest.json \ + --store-root ~/.cache/molmcp/discovery/harness + +Exits 0 whenever a report was produced -- a rejection is a successful +evaluation -- and 1 only when the observation could not be turned into +one. Developer-side and advisory: this is not wired into CI, because a +CI runner has no subagent to start. +""" + +from __future__ import annotations + +import argparse +import json +from collections import Counter +from collections.abc import Mapping, Sequence +from dataclasses import dataclass +from pathlib import Path +from typing import Protocol + +from harness_cases import graduated_ids, held_out_ids + +from molmcp.components import ( + SHA_PATTERN, + GitHubTransport, + ImmutableGitStore, + UnknownShaError, +) +from molmcp.evolution import ( + ContractOutcome, + EvalCase, + EvaluationError, + EvaluationReport, + Metrics, + evaluate, +) + +#: The only observation payload version this adapter reads. +_SCHEMA = "harness-eval/1" + +#: The blind labels an observer may use, and nothing else. +_LABELS: tuple[str, ...] = ("A", "B") + +#: The two roles a label may be unblinded into. +_CHAMPION = "champion" +_CHALLENGER = "challenger" +_ROLES = frozenset({_CHAMPION, _CHALLENGER}) + +#: Observation keys only an unblinded observer could have written. +_LEAKING_KEYS = frozenset( + {"sides", _CHAMPION, _CHALLENGER, "champion_sha", "challenger_sha"} +) + +#: The closed observation schema. Closed rather than merely checked for +#: the banned keys above: the next leak would arrive under a name no +#: list here anticipated. +_OBSERVATION_KEYS = frozenset({"schema", "readings"}) + +#: The closed reading schema, for the same reason. +_READING_KEYS = frozenset( + {"case_id", "seed", "side", "contract_met", "tool_errors", "call_count"} +) + +#: Readings no transcript can support. Permitting the key invites the +#: next observer to guess a number and call it telemetry. +_UNOBSERVABLE_KEYS: tuple[str, ...] = ("tokens", "latency_s") + +#: What the manifest must carry before anything is unblinded. +_MANIFEST_KEYS = frozenset( + { + "champion_sha", + "challenger_sha", + "component", + "affected_paths", + "seeds", + "sides", + } +) + +#: What both sides read for the two unobservable readings. Equal on both +#: sides is the point: the gate compares each reading on its own, so an +#: equal pair can neither reject nor accept a challenger. +_UNREAD_TOKENS = 0 +_UNREAD_LATENCY_S = 0.0 + +#: The case set, split the way the gate's two arguments split it. +_HELD_OUT: tuple[EvalCase, ...] = tuple(EvalCase(id=name) for name in held_out_ids()) +_GRADUATED: tuple[EvalCase, ...] = tuple(EvalCase(id=name) for name in graduated_ids()) +_HELD_OUT_IDS = frozenset(case.id for case in _HELD_OUT) +_KNOWN_IDS = _HELD_OUT_IDS | frozenset(case.id for case in _GRADUATED) + + +class TreeStore(Protocol): + """The one thing :func:`report` asks a component store for. + + Structural on purpose: the adapter reads a single method, and a + parameter typed as the concrete store would hide a rename of it. + """ + + def tree_path(self, sha: str) -> Path: + """Return the published tree for *sha*, or raise if there is none.""" + ... + + +@dataclass(frozen=True, slots=True) +class ObservedChallenger: + """The checkout under evaluation, as the manifest describes it. + + Implements :class:`molmcp.evolution.Challenger`: three names the + gate carries into its report without interpreting any of them. + + Attributes: + sha: Full commit sha of the challenger checkout. + component: Id of the harness component it changes. + affected_paths: Repository paths it touches. + """ + + sha: str + component: str + affected_paths: tuple[str, ...] + + +@dataclass(frozen=True, slots=True) +class ObservedRunner: + """The graduated contract, already settled by the observer. + + Implements :class:`molmcp.evolution.ContractRunner`. The transcripts + were read before this module ran, so ``run`` opens nothing: it looks + up what the observer recorded for the challenger side and reports + it. A case counts as met only when every round met it. + + Attributes: + met: Whether each graduated case was met on the challenger side, + keyed by case id. + """ + + met: Mapping[str, bool] + + def run(self, tree: Path, cases: Sequence[EvalCase]) -> ContractOutcome: + """Report the observed outcome of *cases*. + + Args: + tree: The challenger checkout. Named by the protocol and + never opened here -- the run it describes is already + over, and re-reading the tree would be a second source. + cases: The graduated cases to report on. + + Returns: + One :class:`molmcp.evolution.ContractOutcome`, failing ids + included so the report's readers can name them. + """ + failed = tuple(case.id for case in cases if not self.met[case.id]) + return ContractOutcome(passed=not failed, failed_case_ids=failed) + + +@dataclass(frozen=True, slots=True) +class ObservedReplay: + """The held-out readings, already counted, looked up by side and round. + + Implements :class:`molmcp.evolution.ReplayFn` and keeps that + protocol's frozen convention for telling the sides apart: the + champion arrives as a sha string, the challenger as the tree a + caller already resolved, so ``isinstance(target, Path)`` is the + whole dispatch. No side argument is added -- a second way to say + which side is a second way to get it wrong. + + Attributes: + champion: The champion's reading for each round, keyed by seed. + challenger: The challenger's reading, same shape. + """ + + champion: Mapping[int, Metrics] + challenger: Mapping[int, Metrics] + + def __call__( + self, target: str | Path, cases: Sequence[EvalCase], seed: int + ) -> Metrics: + """Return one side's reading for one round. + + Args: + target: The champion's sha, or the challenger's tree. + cases: The held-out cases. Named by the protocol; the + reading was summed over them before this module ran. + seed: The round to read. + + Returns: + That side's :class:`molmcp.evolution.Metrics` for *seed*. + """ + table = self.challenger if isinstance(target, Path) else self.champion + return table[seed] + + +@dataclass(frozen=True, slots=True) +class _Reading: + """One observer row: a blind side, a round, and what it counted.""" + + case_id: str + seed: int + side: str + contract_met: bool + tool_errors: int + call_count: int + + +@dataclass(frozen=True, slots=True) +class _Plan: + """The manifest, validated: everything the observer was not told.""" + + champion_sha: str + challenger_sha: str + component: str + affected_paths: tuple[str, ...] + seeds: tuple[int, ...] + sides: Mapping[str, str] + + +def _mapping(value: object, what: str) -> Mapping[str, object]: + """Narrow *value* to a string-keyed mapping or refuse it.""" + if not isinstance(value, Mapping): + raise EvaluationError(f"{what} must be an object, not {type(value).__name__}") + return {str(key): item for key, item in value.items()} + + +def _sequence(value: object, what: str) -> Sequence[object]: + """Narrow *value* to a list-like sequence or refuse it.""" + if isinstance(value, str) or not isinstance(value, Sequence): + raise EvaluationError(f"{what} must be a list, not {type(value).__name__}") + return tuple(value) + + +def _text(value: object, what: str) -> str: + """Narrow *value* to a string or refuse it.""" + if not isinstance(value, str): + raise EvaluationError(f"{what} must be text, not {type(value).__name__}") + return value + + +def _whole(value: object, what: str) -> int: + """Narrow *value* to a non-negative integer or refuse it.""" + if isinstance(value, bool) or not isinstance(value, int): + raise EvaluationError( + f"{what} must be a whole number, not {type(value).__name__}" + ) + if value < 0: + raise EvaluationError(f"{what} must not be negative, got {value}") + return value + + +def _flag(value: object, what: str) -> bool: + """Narrow *value* to a boolean or refuse it.""" + if not isinstance(value, bool): + raise EvaluationError( + f"{what} must be true or false, not {type(value).__name__}" + ) + return value + + +def _sha(value: object, what: str) -> str: + """Narrow *value* to a full commit sha or refuse it.""" + sha = _text(value, what) + if SHA_PATTERN.fullmatch(sha) is None: + raise EvaluationError(f"{what} must be a full commit sha, got {sha!r}") + return sha + + +def _cell(cell: tuple[str, int, str]) -> str: + """Name one ``(side, round, case)`` cell the way a refusal should.""" + side, seed, case_id = cell + return f"case {case_id!r} on side {side} in round {seed}" + + +def _cells(cells: Sequence[tuple[str, int, str]]) -> str: + """Name several cells in one refusal.""" + return "; ".join(_cell(cell) for cell in cells) + + +def _reading_of(entry: Mapping[str, object], position: int) -> _Reading: + """Validate and narrow one observation row. + + Args: + entry: The row as the observer wrote it. + position: Index of the row, so a refusal can point at it. + + Returns: + The row as a :class:`_Reading`. + + Raises: + EvaluationError: The row carries a reading no transcript can + show, is not the closed row schema, names a case that is in + no case set entry, or names a side that is not a blind label. + """ + where = f"reading {position}" + unobservable = [key for key in _UNOBSERVABLE_KEYS if key in entry] + if unobservable: + raise EvaluationError( + f"{where} carries {', '.join(unobservable)}, which cannot be " + f"counted off a transcript. Both sides are read as zero here so " + f"that a guessed number can never decide a verdict." + ) + unknown = sorted(set(entry) - _READING_KEYS) + if unknown: + raise EvaluationError(f"{where} carries unknown keys: {', '.join(unknown)}") + missing = sorted(_READING_KEYS - set(entry)) + if missing: + raise EvaluationError(f"{where} is missing keys: {', '.join(missing)}") + + case_id = _text(entry["case_id"], f"{where} case_id") + if case_id not in _KNOWN_IDS: + raise EvaluationError( + f"{where} names case {case_id!r}, which is in no case set entry. " + f"A mistyped id averaged into a mean is worse than a refusal." + ) + side = _text(entry["side"], f"{where} side") + if side not in _LABELS: + raise EvaluationError( + f"{where} names side {side!r}; only the blind labels " + f"{', '.join(_LABELS)} may appear in an observation." + ) + return _Reading( + case_id=case_id, + seed=_whole(entry["seed"], f"{where} seed"), + side=side, + contract_met=_flag(entry["contract_met"], f"{where} contract_met"), + tool_errors=_whole(entry["tool_errors"], f"{where} tool_errors"), + call_count=_whole(entry["call_count"], f"{where} call_count"), + ) + + +def _observed_readings(observation: Mapping[str, object]) -> tuple[_Reading, ...]: + """Validate the observation and narrow its rows. + + Args: + observation: What the observer wrote, straight from its file. + + Returns: + Every row, validated, in the order the observer wrote them. + + Raises: + EvaluationError: The observation names a side, is not the closed + observation schema, carries another schema version, or holds + a row that does not validate. + """ + named = sorted(_LEAKING_KEYS & set(observation)) + if named: + raise EvaluationError( + f"the observation names a side: {', '.join(named)}. An observer " + f"that can say which checkout it read was told which one it was, " + f"and the whole reading rests on it not knowing." + ) + unknown = sorted(set(observation) - _OBSERVATION_KEYS) + if unknown: + raise EvaluationError( + f"the observation carries unknown keys: {', '.join(unknown)}" + ) + missing = sorted(_OBSERVATION_KEYS - set(observation)) + if missing: + raise EvaluationError(f"the observation is missing keys: {', '.join(missing)}") + schema = _text(observation["schema"], "observation schema") + if schema != _SCHEMA: + raise EvaluationError( + f"the observation reads {schema!r}; this adapter reads {_SCHEMA!r}" + ) + rows = _sequence(observation["readings"], "observation readings") + return tuple( + _reading_of(_mapping(row, f"reading {position}"), position) + for position, row in enumerate(rows) + ) + + +def _unblinded_sides(value: object) -> Mapping[str, str]: + """Read the manifest's label-to-role assignment. + + Args: + value: The manifest's ``sides`` entry. + + Returns: + Each blind label mapped onto its role. + + Raises: + EvaluationError: The assignment is not a bijection from the two + blind labels onto the two roles. Anything else leaves a + reading with no side, or two readings with the same one. + """ + sides = _mapping(value, "manifest sides") + roles = { + label: _text(role, f"manifest side {label!r}") for label, role in sides.items() + } + if set(roles) != set(_LABELS) or set(roles.values()) != _ROLES: + raise EvaluationError( + f"manifest sides must assign each of {', '.join(_LABELS)} exactly " + f"one of {', '.join(sorted(_ROLES))}, got {roles!r}. Unblinding " + f"comes from the manifest alone, so it must be unambiguous." + ) + return roles + + +def _plan_of(manifest: Mapping[str, object]) -> _Plan: + """Validate the manifest the orchestrator wrote before the run. + + Args: + manifest: The manifest, straight from its file. + + Returns: + The validated :class:`_Plan`. + + Raises: + EvaluationError: A key is missing, a sha is not a full commit + sha, a round is repeated, or ``sides`` is not a bijection. + """ + missing = sorted(_MANIFEST_KEYS - set(manifest)) + if missing: + raise EvaluationError(f"the manifest is missing keys: {', '.join(missing)}") + seeds = tuple( + _whole(seed, "manifest seed") + for seed in _sequence(manifest["seeds"], "manifest seeds") + ) + if len(set(seeds)) != len(seeds): + raise EvaluationError( + f"the manifest repeats a round: {list(seeds)}. A round counted " + f"twice weights itself twice in the mean." + ) + return _Plan( + champion_sha=_sha(manifest["champion_sha"], "manifest champion_sha"), + challenger_sha=_sha(manifest["challenger_sha"], "manifest challenger_sha"), + component=_text(manifest["component"], "manifest component"), + affected_paths=tuple( + _text(path, "manifest affected path") + for path in _sequence(manifest["affected_paths"], "manifest affected_paths") + ), + seeds=seeds, + sides=_unblinded_sides(manifest["sides"]), + ) + + +def _refuse_incomplete_grid(readings: Sequence[_Reading], seeds: Sequence[int]) -> None: + """Refuse unless every ``(side, round, case)`` appears exactly once. + + Args: + readings: The validated rows. + seeds: The rounds the manifest asked for. + + Raises: + EvaluationError: A cell is repeated, missing, or not one the + manifest asked for. Any of the three silently changes the + denominator of a mean. + """ + counted = Counter((row.side, row.seed, row.case_id) for row in readings) + expected = { + (label, seed, case_id) + for label in _LABELS + for seed in seeds + for case_id in _KNOWN_IDS + } + repeated = sorted(cell for cell, times in counted.items() if times > 1) + if repeated: + raise EvaluationError(f"the observation reads twice: {_cells(repeated)}") + absent = sorted(expected - set(counted)) + if absent: + raise EvaluationError(f"the observation never reads: {_cells(absent)}") + extra = sorted(set(counted) - expected) + if extra: + raise EvaluationError( + f"the observation reads a cell the manifest never asked for: " + f"{_cells(extra)}" + ) + + +def _refuse_abandoned(readings: Sequence[_Reading]) -> None: + """Refuse a held-out round that did not finish. + + Args: + readings: The validated rows. + + Raises: + EvaluationError: A held-out reading has ``contract_met`` false, + named by case, side and round. An unfinished round reads + cheaper than a finished one -- fewer calls, fewer errors -- + so averaging it in would make giving up look like a gain. + """ + for row in readings: + if row.case_id in _HELD_OUT_IDS and not row.contract_met: + raise EvaluationError( + f"{_cell((row.side, row.seed, row.case_id))} did not finish, " + f"and an unfinished round reads cheaper than a finished one. " + f"Graduate the case or fix the harness; do not let it lower " + f"the mean." + ) + + +def _seed_metrics( + readings: Sequence[_Reading], label: str, seeds: Sequence[int] +) -> Mapping[int, Metrics]: + """Sum one side's held-out readings, round by round. + + Graduated cases are left out on purpose: a graduated case is a + correctness contract, not a reading, and letting its cost into the + sum would let the length of a contract case decide a promotion. + + Args: + readings: The validated rows. + label: The blind label of the side to read. + seeds: The rounds to read. + + Returns: + One :class:`molmcp.evolution.Metrics` per round, with both + unobservable readings pinned to zero. + """ + return { + seed: _summed( + [ + row + for row in readings + if row.side == label + and row.seed == seed + and row.case_id in _HELD_OUT_IDS + ] + ) + for seed in seeds + } + + +def _summed(rows: Sequence[_Reading]) -> Metrics: + """One round's reading: the sum over that round's held-out cases.""" + return Metrics( + tool_errors=sum(row.tool_errors for row in rows), + call_count=sum(row.call_count for row in rows), + tokens=_UNREAD_TOKENS, + latency_s=_UNREAD_LATENCY_S, + ) + + +def _contract_met(readings: Sequence[_Reading], label: str) -> Mapping[str, bool]: + """Read one side's graduated outcome, case by case. + + Both sides run the graduated cases -- neither actor knows which case + graduated, and the observer does not know which side is which -- but + the gate runs the contract on the challenger tree alone, so only the + challenger's rows are ever read here. + + Args: + readings: The validated rows. + label: The blind label of the challenger side. + + Returns: + Whether each graduated case was met in every round. + """ + return { + case.id: all( + row.contract_met + for row in readings + if row.side == label and row.case_id == case.id + ) + for case in _GRADUATED + } + + +def report( + observation: Mapping[str, object], + manifest: Mapping[str, object], + *, + store: TreeStore, +) -> EvaluationReport: + """Turn one blind observation into one verdict. + + The observation is checked before the store is touched, because a + payload that names a side is not a blind reading and no amount of + later care recovers one. The manifest then unblinds the labels, the + held-out rows become the readings and the challenger's graduated + rows become the contract; the verdict itself comes wholly from + :func:`molmcp.evolution.evaluate`. + + Args: + observation: What the observer wrote: a schema tag and one row + per ``(side, round, case)``, under blind labels only. + manifest: What the orchestrator wrote before the run and never + showed the observer, including the label-to-role assignment. + store: Component store, asked for both published trees. The only + checkout mechanism here: a report on a tree that was never + published could not be reproduced. + + Returns: + The :class:`molmcp.evolution.EvaluationReport` for this pair. + + Raises: + EvaluationError: The observation names a side; ``sides`` is not + a bijection; a reading carries an unobservable key; a case + id is unknown; a cell is missing, repeated or unasked for; + or a held-out round did not finish. + molmcp.components.UnknownShaError: Either sha is unpublished. + Left to propagate: it is a different failure from a payload + this layer refuses, and swallowing it would report a verdict + on a tree nobody can check out. + """ + readings = _observed_readings(observation) + plan = _plan_of(manifest) + _refuse_incomplete_grid(readings, plan.seeds) + _refuse_abandoned(readings) + + label_of = {role: label for label, role in plan.sides.items()} + store.tree_path(plan.champion_sha) + challenger_tree = store.tree_path(plan.challenger_sha) + + return evaluate( + ObservedChallenger( + sha=plan.challenger_sha, + component=plan.component, + affected_paths=plan.affected_paths, + ), + challenger_tree, + plan.champion_sha, + _HELD_OUT, + _GRADUATED, + runner=ObservedRunner(_contract_met(readings, label_of[_CHALLENGER])), + replay=ObservedReplay( + _seed_metrics(readings, label_of[_CHAMPION], plan.seeds), + _seed_metrics(readings, label_of[_CHALLENGER], plan.seeds), + ), + seeds=plan.seeds, + ) + + +def _rendered(result: EvaluationReport) -> str: + """Lay the report out for a terminal, adding nothing to it.""" + rounds = ", ".join(str(seed) for seed in result.seeds) + contract = "passed" if result.regression_passed else "failed" + return "\n".join( + ( + f"reason: {result.reason}", + f"accepted: {result.accepted}", + f"candidate sha: {result.candidate_sha}", + f"champion sha: {result.champion_sha}", + f"rounds: {rounds}", + f"contract: {contract}", + _readings_line(_CHAMPION, result.champion_metrics), + _readings_line(_CHALLENGER, result.challenger_metrics), + ) + ) + + +def _readings_line(side: str, metrics: Metrics) -> str: + """One side's four readings, in the order the gate compares them.""" + return ( + f"{side + ':':15}tool_errors={metrics.tool_errors} " + f"call_count={metrics.call_count} tokens={metrics.tokens} " + f"latency_s={metrics.latency_s}" + ) + + +def _loaded(path: Path, what: str) -> Mapping[str, object]: + """Read one JSON payload from disk. + + Args: + path: File to read. + what: How to name it in a refusal. + + Returns: + The payload as a mapping. + + Raises: + EvaluationError: The file is not JSON, or is not a JSON object. + """ + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as broken: + raise EvaluationError(f"{what} at {path} is not JSON: {broken}") from broken + return _mapping(payload, f"{what} at {path}") + + +def _parser() -> argparse.ArgumentParser: + """Build the command line: three paths, all required, no defaults.""" + parser = argparse.ArgumentParser( + prog="harness_eval", + description=( + "Turn a blind observation of two harness runs into a verdict " + "from molmcp.evolution.evaluate." + ), + ) + parser.add_argument( + "--observation", + required=True, + type=Path, + help="Observer payload: blind labels and counted readings.", + ) + parser.add_argument( + "--manifest", + required=True, + type=Path, + help="Orchestrator payload: both shas, the rounds, and the sides.", + ) + parser.add_argument( + "--store-root", + required=True, + type=Path, + help="Component store root holding both published trees.", + ) + return parser + + +def main(argv: Sequence[str] | None = None) -> int: + """Read both payloads, print the report, and say whether it was produced. + + Every path is named on the command line and none of them defaults: + a run whose inputs came from somewhere the command line does not + show is a run nobody else can repeat. + + Args: + argv: Command line arguments, or ``None`` to read the process's. + + Returns: + 0 whenever a report was produced -- a rejection is a successful + evaluation, and the reason it carries is the result. 1 only when + the observation could not be turned into a report at all. + """ + args = _parser().parse_args(argv) + store = ImmutableGitStore(args.store_root, GitHubTransport()) + try: + result = report( + _loaded(args.observation, "the observation"), + _loaded(args.manifest, "the manifest"), + store=store, + ) + except EvaluationError as refusal: + print(f"no report: {refusal}") + return 1 + except UnknownShaError as unpublished: + print( + f"no report: the store has published no tree for {unpublished}, " + f"so a report on it could not be reproduced." + ) + return 1 + print(_rendered(result)) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/molmcp/__init__.py b/src/molmcp/__init__.py index 689afd2..6db229f 100644 --- a/src/molmcp/__init__.py +++ b/src/molmcp/__init__.py @@ -1,26 +1,45 @@ -"""MolMCP — multi-plane MCP for MolCrafts (one product per connection).""" +"""MolMCP — molcrafts core with FastMCP-mounted provider planes.""" from __future__ import annotations +import importlib import importlib.metadata -from .client_config import PlaneToggle, resolve_plane_toggles -from .collection import CollectionIndex, ContextPack, SearchHit, SourceBinding -from .config import AppConfig, ConfigurationError, load_config -from .mcp_provider import MolCraftsContextProvider -from .planes import PlaneInfo, known_plane_ids, list_plane_infos, route_task -from .provider import ( - PROVIDER_ENTRY_POINT_GROUP, - Provider, - discover_providers, - provider_available, -) -from .server import create_plane, create_server - __version__ = importlib.metadata.version("molcrafts-molmcp") +#: Public name -> the submodule that defines it. Resolution is deferred so that +#: ``import molmcp`` does not drag ``.server`` — and through it FastMCP, the +#: library that hosts an MCP server — into a process that only wants a leaf +#: such as ``molmcp.provider_worker.protocol``. Membership here mirrors +#: ``__all__`` minus ``__version__``, which is metadata rather than a module. +_LAZY_EXPORTS: dict[str, str] = { + "PlaneToggle": "client_config", + "resolve_plane_toggles": "client_config", + "CollectionIndex": "collection", + "ContextPack": "collection", + "SearchHit": "collection", + "SourceBinding": "collection", + "AppConfig": "config", + "ConfigurationError": "config", + "load_config": "config", + "MolCraftsContextProvider": "mcp_provider", + "CORE_PLANE_ID": "planes", + "PlaneInfo": "planes", + "known_plane_ids": "planes", + "list_plane_infos": "planes", + "route_task": "planes", + "PROVIDER_ENTRY_POINT_GROUP": "provider", + "Provider": "provider", + "discover_providers": "provider", + "provider_available": "provider", + "create_plane": "server", + "create_server": "server", + "create_stack": "server", +} + __all__ = [ "AppConfig", + "CORE_PLANE_ID", "CollectionIndex", "ConfigurationError", "ContextPack", @@ -34,6 +53,7 @@ "__version__", "create_plane", "create_server", + "create_stack", "discover_providers", "known_plane_ids", "list_plane_infos", @@ -42,3 +62,39 @@ "resolve_plane_toggles", "route_task", ] + + +def __getattr__(name: str) -> object: + """Resolve a public name by importing its submodule on first use. + + Args: + name: Attribute requested on the ``molmcp`` package. + + Returns: + The resolved object, cached into the module globals so the import + happens at most once. + + Raises: + AttributeError: If ``name`` is not one of the lazy public exports. + Raising here is load-bearing: CPython's ``_handle_fromlist`` only + falls back to importing a submodule after the package refuses the + attribute, which is what keeps ``from molmcp import cli`` (and + ``settings`` / ``provider`` / ``runtime`` / ``client_config``) + working. + """ + module = _LAZY_EXPORTS.get(name) + if module is None: + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") + value = getattr(importlib.import_module(f".{module}", __name__), name) + globals()[name] = value + return value + + +def __dir__() -> list[str]: + """List the public surface plus whatever has already been resolved. + + Returns: + Sorted attribute names, including every entry of ``__all__`` whether + or not its submodule has been imported yet. + """ + return sorted(set(globals()) | set(__all__)) diff --git a/src/molmcp/cli.py b/src/molmcp/cli.py index 052159e..830d0e3 100644 --- a/src/molmcp/cli.py +++ b/src/molmcp/cli.py @@ -1,4 +1,4 @@ -"""Plane-oriented MolMCP CLI — one MCP process per product plane.""" +"""MolMCP CLI — molcrafts core plus one process per provider plane.""" from __future__ import annotations @@ -10,36 +10,69 @@ from pathlib import Path from typing import Any -from . import settings -from .client_config import render_client +from . import __version__, settings +from .client_config import render_init +from .components import GitError from .config import AppConfig, ConfigurationError, load_config -from .planes import known_plane_ids, list_plane_infos, route_task -from .runtime import build_collection -from .server import create_plane +from .gate import run_gate +from .harness_install import install_harness_components +from .harness_sync import relocate_pointer, rollback_source, sync_source +from .host import ( + HOSTS, + default_write_path, + install_extra_skills, + install_skill, + write_adapter, +) +from .planes import ( + CORE_PLANE_ID, + GONE_PLANE_IDS, + gone_plane_message, + known_plane_ids, + list_plane_infos, + route_task, +) +from .runtime import build_collection, resolved_cache_dir +from .server import create_plane, create_stack def _build_parser() -> argparse.ArgumentParser: parser = argparse.ArgumentParser( prog="molmcp", description=( - "MolCrafts multi-plane MCP: one product domain per connection. " - "Default: enable all planes in the client; use --disable / --enable." + "MolCrafts MCP: `serve` runs the composed core; " + "`init ` wires the host and installs managed skills." ), ) + parser.add_argument( + "-V", + "--version", + action="version", + version=f"%(prog)s {__version__}", + ) commands = parser.add_subparsers(dest="command", required=True) serve = commands.add_parser( "serve", - help="Start one MCP plane (required plane id).", + help="Start the composed molcrafts stack (default) or one plane.", ) _config_argument(serve) serve.add_argument( "plane", + nargs="?", + default=None, help=( - "Plane to serve: catalog | molcrafts | " - "(run `molmcp planes` for the list)." + "Omit to mount enabled providers onto molcrafts (FastMCP namespace). " + "Pass molcrafts or a provider name for a single-plane debug server." ), ) + serve.add_argument( + "--disable", + action="append", + default=[], + metavar="PLANE", + help="Omit a provider mount (written by `molmcp init --disable`).", + ) serve.add_argument( "--transport", choices=["stdio", "streamable-http"], @@ -56,7 +89,7 @@ def _build_parser() -> argparse.ArgumentParser: planes = commands.add_parser( "planes", - help="List connectable MCP planes (on-demand multi-link catalog).", + help="List the molcrafts core and optional provider planes.", ) planes.add_argument( "--json", @@ -70,44 +103,40 @@ def _build_parser() -> argparse.ArgumentParser: ) route.add_argument("task", help="User task description.") - client = commands.add_parser( - "client", + init = commands.add_parser( + "init", help=( - "Emit host MCP config. Default: all planes enabled; " - "use --disable / --enable to toggle." + "Install managed skills (molcrafts, molexp-plan) and MCP " + "config for one host. molcrafts cannot be disabled." ), ) - client.add_argument( + init.add_argument( "host", - nargs="?", - default=None, - choices=["grok", "claude", "cursor"], - help=( - "Where the config is headed. The body is the same standard " - "mcpServers JSON for every host; this only picks the default " - "output path." - ), + # The host list has one home: repeating it here would be a second + # table to keep in step with `molmcp.host.layout.HOSTS`. + choices=tuple(HOSTS), + help="Host to wire (user-level skills + MCP JSON).", ) - client.add_argument( + init.add_argument( "--enable", action="append", default=[], metavar="PLANE", - help="Enable a plane (after --disable). Repeatable.", + help="Enable a provider mount (after --disable). Repeatable.", ) - client.add_argument( + init.add_argument( "--disable", action="append", default=[], metavar="PLANE", - help="Disable a plane. Repeatable. Default is all enabled.", + help="Omit a provider mount. Repeatable.", ) - client.add_argument( + init.add_argument( "-o", "--output", type=Path, default=None, - help="Write to this path (default: print to stdout).", + help="MCP JSON path (default: that host's user config).", ) info = commands.add_parser("info", help="Show registry and index coverage.") @@ -158,6 +187,92 @@ def _build_parser() -> argparse.ArgumentParser: _scope_arguments(config_remove) config_remove.add_argument("key") config_remove.add_argument("value", nargs="?", default=None) + config_harness = config_actions.add_parser( + "harness", + help="Author the named harness sources this install fetches from.", + ) + harness_actions = config_harness.add_subparsers( + dest="harness_action", required=True + ) + harness_set = harness_actions.add_parser( + "set", + help="Upsert one harness source, addressed by a locator.", + ) + _scope_arguments(harness_set) + harness_set.add_argument( + "locator", + help=( + "Origin as you write it: owner/repo, a GitHub URL, or a ~/ " + "or absolute checkout. The same origin upserts the same entry." + ), + ) + harness_set.add_argument( + "--alias", + default=None, + help="Name this source; omit to default to origin on insert.", + ) + harness_set.add_argument( + "--enable", + action="append", + default=[], + metavar="TOKEN", + help="Bundle to enable. Repeatable. 'all' means every bundle.", + ) + harness_set.add_argument( + "--disable", + action="append", + default=[], + metavar="TOKEN", + help="Bundle to disable. Repeatable. 'all' leaves the source with none.", + ) + harness_remove = harness_actions.add_parser( + "remove", + help="Drop the harness source matching an alias or locator.", + ) + _scope_arguments(harness_remove) + harness_remove.add_argument( + "name", + help="The entry's alias, or any accepted locator spelling of its origin.", + ) + + # A second top-level verb rather than a `config harness` leaf: `config` + # edits the settings file and stops there, while this one reaches the + # network (or a checkout), writes into the shared store and moves an + # activation pointer. Putting a fetch behind `molmcp config` would make a + # settings edit and a fetch look like the same kind of act. + harness_cmd = commands.add_parser( + "harness", + help="Fetch and activate the harness sources this install names.", + ) + harness_verbs = harness_cmd.add_subparsers(dest="harness_verb", required=True) + harness_sync = harness_verbs.add_parser( + "sync", + help="Resolve one named source's ref, publish that commit, activate it.", + ) + _config_argument(harness_sync) + harness_sync.add_argument( + "name", + help=( + "The harness source to sync: its alias, or any accepted locator " + "spelling of its origin. No default: with several sources " + "configured, guessing one would fetch code the operator did " + "not ask for." + ), + ) + harness_rollback = harness_verbs.add_parser( + "rollback", + help="Activate the commit this source's last sync displaced.", + ) + _config_argument(harness_rollback) + harness_rollback.add_argument( + "name", + help=( + "The harness source to roll back: its alias, or any accepted " + "locator spelling of its origin. No default, for the reason " + "`sync` has none: with several sources configured, guessing " + "one would change what a plane serves without being asked." + ), + ) cache = commands.add_parser( "cache", @@ -180,6 +295,13 @@ def _build_parser() -> argparse.ArgumentParser: help="Drop cached snapshots for sources that are no longer configured.", ) + # No flags, deliberately. There is one profile, so there is nothing to + # select; a required check with an off switch is not a required check. + commands.add_parser( + "gate", + help="Check the wiring contract this repository's required check runs.", + ) + return parser @@ -235,29 +357,39 @@ def _optional(values: list[str]) -> list[str] | None: def _serve(args: argparse.Namespace) -> int: - plane = args.plane.strip().lower() - known = known_plane_ids() - # Allow serving any discovered provider even if not in the static meta table. - if plane not in known and plane not in {p.name for p in _discover_safe()}: - raise ConfigurationError( - f"unknown plane {plane!r}. Run `molmcp planes` for the catalog." - ) - - config = _load(args) if plane == "molcrafts" else None - # Provider planes may still load config for HTTP auth settings. - if plane not in {"catalog", "molcrafts"}: - try: - config = _load(args) - except ConfigurationError: - config = None - except FileNotFoundError: - config = None + plane_raw = args.plane + plane = plane_raw.strip().lower() if plane_raw else None + if plane in GONE_PLANE_IDS: + raise ConfigurationError(gone_plane_message(plane)) + if plane is not None: + known = known_plane_ids() + if plane not in known and plane not in {p.name for p in _discover_safe()}: + raise ConfigurationError( + f"unknown plane {plane!r}. Run `molmcp planes` for the list." + ) - server = create_plane( - plane, - config=config, - discover_entry_points=not args.no_discover, - ) + config = None + try: + config = _load(args) + except (ConfigurationError, FileNotFoundError): + config = None + + if plane is None: + server = create_stack( + config=config, + disable=args.disable or (), + discover_entry_points=not args.no_discover, + ) + else: + if args.disable: + raise ConfigurationError( + "--disable applies to composed `molmcp serve` only" + ) + server = create_plane( + plane, + config=config, + discover_entry_points=not args.no_discover, + ) transport = args.transport or ( config.server.transport if config is not None else "stdio" ) @@ -288,21 +420,23 @@ def _planes(args: argparse.Namespace) -> int: planes = [p.to_dict() for p in list_plane_infos()] payload = { "ok": True, - "model": "multi-plane-default-all", + "core": CORE_PLANE_ID, + "model": "molcrafts core + optional provider planes", "planes": planes, "hint": ( - "Default: enable every plane in the client. " - "molmcp client grok # all on\n" - "molmcp client grok --disable molq # all except molq\n" - "molmcp client grok --disable molq --enable molq # re-enable" + "`molmcp serve` mounts enabled providers onto molcrafts. " + "Disable a mount with:\n" + "molmcp init grok --disable molq\n" + "molmcp init grok --disable molq --enable molq # re-enable" ), } if args.json: _emit(payload) return 0 - print("MolCrafts MCP planes (default: all enabled in client):\n") + print("MolCrafts MCP — molcrafts core (always on) + provider planes:\n") for row in planes: - print(f" {row['id']:12} {row['serve_command']}") + flag = "core" if not row.get("disableable", True) else "optional" + print(f" {row['id']:12} {row['serve_command']} [{flag}]") print(f" {row['purpose']}") print(f" when: {row['when_to_connect']}") if row.get("tools_hint"): @@ -317,29 +451,76 @@ def _route(args: argparse.Namespace) -> int: return 0 -def _client(args: argparse.Namespace) -> int: - toggle, text = render_client( +def _init(args: argparse.Namespace) -> int: + """Wire one host: MCP JSON, usage skill, bundles, adapter, catalog components. + + MCP (Model Context Protocol) is the wire protocol an AI client uses to + call tools, so the JSON written here is that client's list of servers to + launch. Every other destination belongs to :mod:`molmcp.host`, whose write + primitives are composed here in order rather than hidden behind a facade, + so each destination has exactly one visible writer. + + ``--source`` is interpreted once, by ``resolve_bundle_source``, and it is + that resolved value — never the raw flag — that the three bundle + primitives receive. A checkout that is not a directory therefore fails + here instead of degrading silently to the packaged backend. + + ``install_harness_components`` is the other origin — the commit a + ``molmcp harness sync`` activated, read down to the files its catalog + declares — and it comes **last** for a reason that is not cosmetic. The + placement seam keeps a catalog off the managed usage skill by *skipping* + any destination inside that directory, and skipping protects a file only + once it is there: run before ``install_skill``, the refusal would still + fire and the constitution would then be written over whatever the catalog + had put in its place. + + Args: + args: Parsed ``init`` arguments: the host, the plane toggles + (``--enable`` / ``--disable``, a *plane* being one product's MCP + server), ``-o/--output``, and ``--source``. + + Returns: + ``0`` once the MCP JSON, the usage skill, and the adapter are written, + along with whichever daily and dev files the resolved checkout + supplied — none of them when there is no checkout — and whichever + components the activated harness commits declared, none of them when + no configured source is synced. + + Raises: + FileNotFoundError: If ``--source`` is not a directory, or if a harness + catalog declares a file its own published tree does not hold. + ValueError: If the host or a plane toggle is unknown, or if an + activated commit's catalog cannot be served. + ConfigurationError: If a harness source's pointer names a commit with + no published tree. + """ + toggle, text = render_init( args.host, enable=args.enable, disable=args.disable, ) - if args.output is not None: - path = args.output.expanduser() - path.parent.mkdir(parents=True, exist_ok=True) - path.write_text(text, encoding="utf-8") - print( - f"wrote {path} enabled={list(toggle.enabled)} " - f"disabled={list(toggle.disabled)}", - file=sys.stderr, - ) - return 0 - # stderr summary so piping stdout stays clean + path = ( + args.output.expanduser() + if args.output is not None + else default_write_path(args.host) + ) + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + skill_path = install_skill(args.host) + extra_paths = install_extra_skills(args.host) + adapter_path = write_adapter(args.host) + placed = install_harness_components(args.host) + extra_lines = "".join(f"wrote {p}\n" for p in extra_paths) print( - f"# enabled: {', '.join(toggle.enabled)}" - + (f" # disabled: {', '.join(toggle.disabled)}" if toggle.disabled else ""), + f"wrote {path} enabled={list(toggle.enabled)} " + f"disabled={list(toggle.disabled)}\n" + f"wrote {skill_path}\n" + f"{extra_lines}" + f"wrote {adapter_path}\n" + f"placed {len(placed.installed)} harness catalog component file(s), " + f"{len(placed.skipped)} refused", file=sys.stderr, ) - sys.stdout.write(text) return 0 @@ -412,6 +593,22 @@ def _config(args: argparse.Namespace) -> int: a plane server inherits its working directory from whichever MCP client launched it, so a project-scoped default would make configuration depend on an accident. + + The branch chain is exhaustive by construction: an action with no + branch raises rather than falling through to ``remove_value``, which + would delete a setting nobody asked to delete. + + Args: + args: The parsed ``config`` namespace, carrying ``config_action`` + and whichever arguments that action's subparser declares. + + Returns: + ``0`` once the read is printed or the write is on disk. + + Raises: + ConfigurationError: If ``config_action`` names an action this + handler does not dispatch. + settings.SettingsError: If the settings layer refuses the write. """ if args.config_action == "list": _emit(settings.load_settings(Path.cwd()).to_dict()) @@ -429,13 +626,141 @@ def _config(args: argparse.Namespace) -> int: settings.set_value(target, args.key, args.value) elif args.config_action == "add": settings.add_value(target, args.key, args.value) - else: + elif args.config_action == "harness": + _config_harness(args, target) + elif args.config_action == "remove": settings.remove_value(target, args.key, args.value) + else: + raise ConfigurationError( + f"unrecognized `molmcp config` action: {args.config_action!r}" + ) print(f"wrote {target}", file=sys.stderr) _emit(settings.read_settings_file(target)) return 0 +def _config_harness(args: argparse.Namespace, target: Path) -> None: + """Author one entry of the ``harness`` list, addressed by a locator. + + The string verbs cannot reach this key — ``set`` refuses the bare + member of an object list and no dotted path into an entry exists — so + these two leaves are its only authoring route. They hold their own + branches here rather than inside :func:`_config` so that neither chain + has to nest. + + A locator already in *target* whose ``--alias`` differs from the + stored name is renamed through + :func:`~molmcp.harness_sync.relocate_pointer`, so the activation + pointer follows. Every other write is + :func:`settings.set_harness_source`. This handler does not import + the locator parser or the pointer namer. + + Args: + args: The parsed namespace, carrying ``harness_action`` and — + on the ``set`` leaf — ``locator``, ``alias``, ``enable`` and + ``disable``. + target: The settings file the scope flags selected. + + Raises: + ConfigurationError: If ``harness_action`` names a leaf this + handler does not implement. + settings.SettingsError: If the settings layer refuses the write. + """ + # Read as a bare attribute, never getattr(args, "harness_action", None): + # tests/test_cli_config.py::test_every_registered_config_action_is_dispatched + # calls _config(Namespace(config_action="harness")) with nothing else set and + # treats only ConfigurationError as "this action is unwired". A getattr default + # would fall through to the terminal raise below and report harness as unwired, + # turning a green drift guard red. The bare access raises AttributeError, which + # that test swallows by design. + if args.harness_action == "set": + locator = args.locator + alias = args.alias + enable = tuple(args.enable) + disable = tuple(args.disable) + matched = settings.match_harness_source(_harness_file_sources(target), locator) + if matched is not None and alias is not None and alias != matched.name: + relocate_pointer( + load_config(None), + target, + locator=locator, + name=alias, + enable=enable, + disable=disable, + ) + return + settings.set_harness_source( + target, + locator, + alias=alias, + enable=enable, + disable=disable, + ) + return + if args.harness_action == "remove": + settings.remove_harness_source(target, args.name) + return + raise ConfigurationError( + f"unrecognized `molmcp config harness` action: {args.harness_action!r}" + ) + + +def _harness_file_sources(path: Path) -> tuple[settings.HarnessSource, ...]: + """The ``harness`` entries stored in one settings file, or none.""" + raw = settings.read_settings_file(path) + entries = raw.get("harness", []) + if not isinstance(entries, list): + return () + return tuple( + settings.HarnessSource(**entry) for entry in entries if isinstance(entry, dict) + ) + + +def _harness(args: argparse.Namespace) -> int: + """Dispatch one ``molmcp harness`` verb and report what it did. + + The work belongs to :mod:`molmcp.harness_sync` — :func:`~molmcp. + harness_sync.sync_source` forward along the pointer and + :func:`~molmcp.harness_sync.rollback_source` back along it; this handler + resolves the configuration, hands over the name, and turns the report into + lines. Every failure leaves here as an exception for ``main``'s single + funnel to render, so an operator of a half-configured install gets one + sentence rather than a traceback. + + ``rollback`` prints no tree because it publishes none: it moves a pointer + onto a commit already in the store, so the pointer file and the commit are + the whole of what changed. + + Args: + args: The parsed ``harness`` namespace, carrying ``harness_verb`` and + — on both leaves — ``name`` plus the standard ``--config`` / + ``--env`` pair. + + Returns: + ``0`` once the pointer names the commit that was asked for. + + Raises: + ConfigurationError: If ``harness_verb`` names a verb this handler does + not dispatch, or if the sync or rollback itself refuses the + request. + """ + if args.harness_verb == "sync": + report = sync_source(_load(args), args.name) + state = "activated" if report.promoted else "already activated" + print(f"{report.source}: {report.sha} {state}") + print(f" tree {report.tree}") + print(f" pointer {report.pointer}") + return 0 + if args.harness_verb == "rollback": + rolled = rollback_source(_load(args), args.name) + print(f"{rolled.source}: rolled back to {rolled.sha}") + print(f" pointer {rolled.pointer}") + return 0 + raise ConfigurationError( + f"unrecognized `molmcp harness` verb: {args.harness_verb!r}" + ) + + def _cache_hint( vacuum_report: dict[str, Any] | None, size: int, used: int ) -> str | None: @@ -467,9 +792,7 @@ def _cache(args: argparse.Namespace) -> int: vacuum_report: dict[str, Any] | None = None config = _load(args) - discovery = DiscoveryConfig( - cache_dir=config.cache_dir or DiscoveryConfig().cache_dir - ) + discovery = DiscoveryConfig(cache_dir=resolved_cache_dir(config)) gc_report: dict[str, Any] | None = None if args.gc: gc_report = SnapshotCache(discovery).collect_out_of_scope( @@ -540,6 +863,33 @@ def _cache(args: argparse.Namespace) -> int: return 0 +def _gate(args: argparse.Namespace) -> int: + """Report whether the working directory's wiring contract still holds. + + The verdict has one owner, :func:`molmcp.gate.run_gate`. This handler + reads ``ok`` off the report instead of re-deriving it from ``failed``: + two derivations of one verdict are two things that can later disagree + about the single required check. Each reported disagreement already + names its file and its offending token, so they are printed as handed + over rather than reworded here. + + Args: + args: Parsed ``gate`` arguments. The subcommand carries no flags, + so nothing is read from it; it is taken to keep every handler + one shape. + + Returns: + ``0`` when the report is ok, ``1`` otherwise. + """ + report = run_gate(root=Path.cwd()) + for message in report.failed: + print(f"molmcp: {message}", file=sys.stderr) + if report.ok: + print("wiring contract holds") + return 0 + return 1 + + def main(argv: list[str] | None = None) -> int: arguments = list(sys.argv[1:] if argv is None else argv) if not arguments: @@ -550,13 +900,15 @@ def main(argv: list[str] | None = None) -> int: "serve": _serve, "planes": _planes, "route": _route, - "client": _client, + "init": _init, "info": _info, "search": _search, "explore": _explore, "index": _index, "config": _config, + "harness": _harness, "cache": _cache, + "gate": _gate, } try: return handlers[args.command](args) @@ -569,6 +921,15 @@ def main(argv: list[str] | None = None) -> int: # the CLI owes the user a sentence, not a traceback. sqlite3.Error, OSError, + # So is a ref that does not resolve or a repository that will not + # answer. GitError is registered here rather than converted at the + # verb that raised it, for two reasons: it is a RuntimeError, so it + # is caught by nothing above and would otherwise escape as a + # traceback; and its message already names the ref, the coordinate + # or the checkout root that git could not answer for, which a + # rewrite into ConfigurationError would replace with a guess about + # which of them was wrong. + GitError, ) as exc: print(f"molmcp: {exc}", file=sys.stderr) return 2 diff --git a/src/molmcp/client_config.py b/src/molmcp/client_config.py index 6d32923..210d403 100644 --- a/src/molmcp/client_config.py +++ b/src/molmcp/client_config.py @@ -1,4 +1,22 @@ -"""Generate host MCP client configs — default all planes, --enable/--disable.""" +"""Generate host MCP client configs — core always on, providers togglable. + +MCP (Model Context Protocol) is the wire protocol an AI client uses to call +tools; a *host* is one such client, and its MCP JSON is the file listing the +servers it should launch. This module is where that JSON body is decided, and +the functions defined here return text rather than writing it — the ``init`` +command in :mod:`molmcp.cli` is what puts it on disk. + +Where the file lands, and every other file ``molmcp init`` writes, is owned +by :mod:`molmcp.host`. The five names imported from there below are the ones +this module's own signatures need, re-exported as the very same objects and +never copies, so the one host path table stays in ``molmcp.host.layout``. + +The primitives that *write* those destinations — ``install_skill`` and its +siblings — are deliberately absent. They have a single importable home, +:mod:`molmcp.host`; a second spelling here would be a second name to keep in +step with it, and the file that copies the usage constitution should have one +caller-visible source. +""" from __future__ import annotations @@ -7,12 +25,29 @@ import shutil import sys from dataclasses import dataclass -from pathlib import Path -from typing import Any, Literal -from .planes import list_plane_infos - -Host = Literal["grok", "claude", "cursor"] +# Re-exported, not used here: nothing in this module resolves a path, since +# home is joined in ``molmcp.host``. The name stays because it is what the +# test suite patches to move ``Path.home()`` off the developer's real home, +# and it is the very ``pathlib.Path`` class that ``molmcp.host`` joins its +# layout tuples against — so patching it here redirects the writer too. +from pathlib import Path as Path +from typing import Any + +from .host import ( + HOSTS, + SKILL_NAME, + Host, + default_write_path, + layout_for, +) +from .planes import ( + CORE_PLANE_ID, + GONE_PLANE_IDS, + core_disable_message, + gone_plane_message, + list_plane_infos, +) @dataclass(frozen=True, slots=True) @@ -32,17 +67,23 @@ def to_dict(self) -> dict[str, Any]: def default_plane_ids() -> tuple[str, ...]: - """Planes with installed deps (catalog, molcrafts, then α). + """Core plus provider planes with installed deps. Optional science packages that are not installed are omitted silently — no pytest-style skip; they simply never appear in client configs. + ``molcrafts`` is always first. """ infos = list_plane_infos(include_unavailable_providers=False) ids = [p.id for p in infos] - # Prefer catalog → molcrafts first, then the rest sorted. - head = [x for x in ("catalog", "molcrafts") if x in ids] - tail = sorted(x for x in ids if x not in head) - return tuple(head + tail) + tail = sorted(x for x in ids if x != CORE_PLANE_ID) + return (CORE_PLANE_ID, *tail) + + +def _ensure_core(planes: tuple[str, ...]) -> tuple[str, ...]: + if CORE_PLANE_ID in planes: + tail = tuple(p for p in planes if p != CORE_PLANE_ID) + return (CORE_PLANE_ID, *tail) + return (CORE_PLANE_ID, *planes) def resolve_plane_toggles( @@ -51,33 +92,43 @@ def resolve_plane_toggles( disable: list[str] | tuple[str, ...] = (), available: tuple[str, ...] | None = None, ) -> PlaneToggle: - """Default: all planes on. Apply ``--disable`` then ``--enable``. + """Default: core + every provider on. Apply ``--disable`` then ``--enable``. + + ``molcrafts`` cannot be disabled. Retired ids such as ``catalog`` error. Raises: - ValueError: unknown plane id in enable/disable. + ValueError: unknown plane id, retired plane, or attempt to disable core. """ - all_planes = available if available is not None else default_plane_ids() + all_planes = _ensure_core( + available if available is not None else default_plane_ids() + ) known = set(all_planes) enabled = set(all_planes) def _norm(name: str) -> str: return name.strip().lower() - for raw in disable: - plane = _norm(raw) + def _check(plane: str) -> None: + if plane in GONE_PLANE_IDS: + raise ValueError(gone_plane_message(plane)) + if plane == CORE_PLANE_ID: + return if plane not in known: raise ValueError(f"unknown plane {plane!r}; known: {', '.join(all_planes)}") + + for raw in disable: + plane = _norm(raw) + _check(plane) + if plane == CORE_PLANE_ID: + raise ValueError(core_disable_message()) enabled.discard(plane) for raw in enable: plane = _norm(raw) - if plane not in known: - raise ValueError(f"unknown plane {plane!r}; known: {', '.join(all_planes)}") + _check(plane) enabled.add(plane) - if not enabled: - raise ValueError("at least one plane must remain enabled") - + enabled.add(CORE_PLANE_ID) ordered = tuple(p for p in all_planes if p in enabled) disabled = tuple(p for p in all_planes if p not in enabled) return PlaneToggle(enabled=ordered, disabled=disabled, all_planes=all_planes) @@ -103,31 +154,37 @@ def _molmcp_command() -> list[str]: return [sys.executable, "-m", "molmcp"] -def serve_argv(plane: str) -> list[str]: - return [*_molmcp_command(), "serve", plane] +def serve_argv(plane: str | None = None, *, disable: tuple[str, ...] = ()) -> list[str]: + """Argv for one host spawn. + ``plane is None`` is the composed stack (``molmcp serve``). Provider + disables are forwarded as ``--disable`` so the child process omits those + FastMCP mounts. A named *plane* is the single-plane debug server. + """ + parts = [*_molmcp_command(), "serve"] + if plane is not None: + parts.append(plane) + return parts + for name in disable: + parts.extend(["--disable", name]) + return parts -def render_mcp_json(toggle: PlaneToggle) -> dict[str, Any]: - """The standard ``mcpServers`` map, listing only the enabled planes. - Every host molmcp targets reads this shape: Claude Code and Cursor - natively, and Grok alongside its own ``config.toml`` (from - ``~/.claude.json``, ``.cursor/mcp.json`` and project ``.mcp.json``). +def render_mcp_json(toggle: PlaneToggle) -> dict[str, Any]: + """One ``mcpServers`` entry: composed ``molmcp serve``. - A disabled plane is simply absent. The TOML renderer this replaces - emitted every plane with ``enabled = false``, which only that one - format understood. + Disabled providers become ``--disable`` flags on that command. Every host + molmcp targets reads this JSON shape. """ - cmd = _molmcp_command() + cmd = serve_argv(disable=toggle.disabled) return { "mcpServers": { - plane: {"command": cmd[0], "args": cmd[1:] + ["serve", plane]} - for plane in toggle.enabled + CORE_PLANE_ID: {"command": cmd[0], "args": cmd[1:]}, } } -def render_client( +def render_init( host: Host | None = None, *, enable: list[str] | tuple[str, ...] = (), @@ -136,42 +193,39 @@ def render_client( ) -> tuple[PlaneToggle, str]: """Return ``(toggle, config text)``. - ``host`` selects only where the result is meant to go; the body is the - same JSON for all of them. - """ - if host is not None and host not in _HOST_PATHS: - raise ValueError( - f"unknown host {host!r}; known: {', '.join(sorted(_HOST_PATHS))}" - ) - toggle = resolve_plane_toggles(enable=enable, disable=disable, available=available) - return toggle, json.dumps(render_mcp_json(toggle), indent=2) + "\n" + The body is the same JSON for every host, so *host* is validated here and + used for nothing else: this function returns text, not a destination. + Callers get the path from :func:`~molmcp.host.default_write_path`. + Args: + host: A key of :data:`~molmcp.host.HOSTS`, or ``None`` to render the + body without naming a destination. + enable: Provider planes to switch back on, applied after *disable*. + disable: Provider planes to leave unmounted. + available: Plane ids to choose from; defaults to the installed ones. -#: Where each host expects to find the JSON, relative to home unless noted. -_HOST_PATHS: dict[str, tuple[str, ...]] = { - # Claude Code merges the user file; Cursor and Grok read project files. - "claude": (".claude.json",), - "cursor": (".cursor", "mcp.json"), - # Grok reads project .mcp.json below its own config.toml in priority. - "grok": (".mcp.json",), -} + Returns: + The resolved :class:`PlaneToggle` and the MCP JSON text to write. - -def default_write_path(host: Host) -> Path: - """Conventional destination for *host*'s MCP config.""" - if host not in _HOST_PATHS: - raise ValueError( - f"unknown host {host!r}; known: {', '.join(sorted(_HOST_PATHS))}" - ) - return Path.home().joinpath(*_HOST_PATHS[host]) + Raises: + ValueError: If *host* is not a known host, or a plane toggle is not + resolvable. + """ + if host is not None and host not in HOSTS: + raise ValueError(f"unknown host {host!r}; known: {', '.join(sorted(HOSTS))}") + toggle = resolve_plane_toggles(enable=enable, disable=disable, available=available) + return toggle, json.dumps(render_mcp_json(toggle), indent=2) + "\n" __all__ = [ + "HOSTS", "Host", "PlaneToggle", + "SKILL_NAME", "default_plane_ids", "default_write_path", - "render_client", + "layout_for", + "render_init", "render_mcp_json", "resolve_plane_toggles", "serve_argv", diff --git a/src/molmcp/components/__init__.py b/src/molmcp/components/__init__.py new file mode 100644 index 0000000..336762e --- /dev/null +++ b/src/molmcp/components/__init__.py @@ -0,0 +1,99 @@ +"""Stdlib shared leaf: harness catalog types and GitHub HTTP transport. + +This package is not a new architecture layer. Outer and inner modules +import it; it imports only the standard library (plus relative siblings). +It is not re-exported from :mod:`molmcp`. + +The catalog half reads one checkout's ``harness.toml`` into frozen types. +A *harness catalog* lists installable pieces and named groups of those +pieces. Identity is the commit *SHA* (Secure Hash Algorithm fingerprint: +40 lowercase hex characters) the caller passes in; the TOML file must +not contain a ``sha`` key. Catalog loading does not inspect git. + +Two catalog checks, in order, and they are not the same: + +* *Language gate* — the file must match the catalog grammar (known + keys, known kinds, ``requires`` tokens drawn only from + ``ALLOWED_REQUIRES``). +* *Eligibility* — every ``requires`` token that survived the language + gate must also be one the caller currently supports + (``supported_capabilities``). An unknown token still fails the + language gate even if the caller listed it as supported. + +A *component* is one installable piece. ``ComponentKind`` is the enum +of the five kinds (``skill``, ``agent``, ``rule``, ``provider``, +``overlay``). A *bundle* is a named grouping of component ids; it is +not a ``ComponentKind``. An *entrypoint* is a ``module:object`` string +stored for a later import; this package never imports it. + +The git half is :class:`GitTransport` with its two implementations plus +:func:`extract_git_archive`. :class:`GitHubTransport` reaches a coordinate +over stdlib ``urllib``, with an optional GitHub personal access token the +caller supplies; :class:`LocalGitTransport` reaches a checkout already on +disk by running ``git`` there, and opens no socket. This package never +reads the environment. + +The store half is :class:`ImmutableGitStore`. A *SHA directory* is +``/commits//`` with ``metadata.json`` plus ``tree/``. +*Flatten the inner tree* means installing the tarball's single +top-level directory (what :func:`extract_git_archive` returns) as that +``tree/``, so ``harness.toml`` sits at the catalog root, not under +``-/``. + +The activation half is :class:`Activation`. :meth:`Activation.bind` is +the only constructor: it loads the pointer file or an empty in-memory +record and does not write. The pointer names three published SHAs — +*current*, *staged*, and *previous*. ``IneligibleShaError`` is +``stage`` refusing a SHA that has no complete tree or whose catalog +the caller cannot honor. +""" + +from .activate import Activation +from .catalog import HarnessCatalog, ResolvedBundle, load_harness_catalog +from .git import ( + GitError, + GitHubTransport, + GitTransport, + LocalGitTransport, + extract_git_archive, +) +from .models import ( + ALLOWED_REQUIRES, + COMPONENT_NAME_PATTERN, + KIND_PATH_PREFIX, + SHA_PATTERN, + BundleSpec, + CatalogError, + ComponentKind, + ComponentSpec, +) +from .store import ( + ImmutableGitStore, + ShaConflictError, + StoreError, + UnknownShaError, +) + +__all__ = [ + "ALLOWED_REQUIRES", + "Activation", + "BundleSpec", + "COMPONENT_NAME_PATTERN", + "CatalogError", + "ComponentKind", + "ComponentSpec", + "GitError", + "GitHubTransport", + "GitTransport", + "HarnessCatalog", + "ImmutableGitStore", + "KIND_PATH_PREFIX", + "LocalGitTransport", + "ResolvedBundle", + "SHA_PATTERN", + "ShaConflictError", + "StoreError", + "UnknownShaError", + "extract_git_archive", + "load_harness_catalog", +] diff --git a/src/molmcp/components/activate.py b/src/molmcp/components/activate.py new file mode 100644 index 0000000..19b074b --- /dev/null +++ b/src/molmcp/components/activate.py @@ -0,0 +1,292 @@ +"""Activation pointer: current, previous, and staged SHA on disk. + +The only public constructor is :meth:`Activation.bind`. Each mutation +reloads the frozen record from the pointer file, writes a new record +atomically, then refreshes the instance properties from what was +written. Catalog eligibility is checked on ``stage``; this module does +not own a capability universe. +""" + +from __future__ import annotations + +import json +import os +from dataclasses import dataclass +from pathlib import Path + +from .catalog import CatalogError, load_harness_catalog +from .store import ImmutableGitStore + +#: Version field written into the pointer JSON; unknown values raise +#: ActivationVersionError. +ACTIVATION_VERSION = 1 +_POINTER_KEYS = frozenset({"version", "active", "staging", "previous"}) + + +class ActivationError(Exception): + """Base error for :class:`Activation` operations.""" + + +class ActivationVersionError(ActivationError): + """Raised when the pointer file is not a version-1 activation record. + + Unknown ``version``, extra or missing fields, and invalid JSON all + fail here. A missing file is not an error; it is an empty record. + """ + + +class IneligibleShaError(ActivationError): + """Raised when ``stage`` cannot accept a SHA. + + ``stage`` maps two failures onto this type and does not raise + :class:`~molmcp.components.store.UnknownShaError`: (1) + ``store.has(sha)`` is false (no complete SHA directory); (2) + :func:`load_harness_catalog` raises :class:`CatalogError` (invalid + ``harness.toml``, or a ``requires`` token this process cannot + honor). The pointer file is left unchanged. + """ + + +class NothingStagedError(ActivationError): + """Raised when ``promote`` runs with no staged SHA.""" + + +class NothingToRollbackError(ActivationError): + """Raised when ``rollback`` runs with no previous SHA.""" + + +@dataclass(frozen=True, slots=True) +class _ActivationRecord: + current: str | None + previous: str | None + staged: str | None + + +def _empty_record() -> _ActivationRecord: + return _ActivationRecord(current=None, previous=None, staged=None) + + +def _optional_sha(value: object, field: str) -> str | None: + if value is None: + return None + if isinstance(value, str): + return value + raise ActivationVersionError(f"pointer field {field!r} must be a string or null") + + +def _record_from_payload(payload: object) -> _ActivationRecord: + if not isinstance(payload, dict): + raise ActivationVersionError("activation pointer is not an object") + if set(payload) != _POINTER_KEYS: + raise ActivationVersionError("activation pointer has unknown or missing fields") + if payload["version"] != ACTIVATION_VERSION: + raise ActivationVersionError( + f"unsupported activation version {payload['version']!r}" + ) + return _ActivationRecord( + current=_optional_sha(payload["active"], "active"), + previous=_optional_sha(payload["previous"], "previous"), + staged=_optional_sha(payload["staging"], "staging"), + ) + + +def _load_record(path: Path) -> _ActivationRecord: + if not path.is_file(): + return _empty_record() + try: + payload = json.loads(path.read_text(encoding="utf-8")) + except json.JSONDecodeError as exc: + raise ActivationVersionError("activation pointer is not valid JSON") from exc + return _record_from_payload(payload) + + +def _write_record(path: Path, record: _ActivationRecord) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + temp = path.with_name(f"{path.name}.partial") + payload = { + "version": ACTIVATION_VERSION, + "active": record.current, + "staging": record.staged, + "previous": record.previous, + } + temp.write_text(json.dumps(payload), encoding="utf-8") + os.replace(temp, path) + + +class Activation: + """Read-only view of the activation pointer, mutated only via methods. + + Construct only via :meth:`bind`; ``Activation(...)`` raises + ``TypeError``. The pointer is a JSON file of three SHA names, not a + copy of the trees: + + * *current* (JSON ``active``): SHA now in effect + * *staged* (JSON ``staging``): SHA that passed ``stage`` and is + waiting for ``promote``; does not change current + * *previous* (JSON ``previous``): SHA ``rollback`` would restore + (one-level; a new promote overwrites it) + + ``stage`` sets staged only. ``promote`` does staged→current, + current→previous, staged=None. ``rollback`` does previous→current, + previous=None, staged unchanged. + """ + + __slots__ = ("_path", "_record", "_store", "_supported_capabilities") + + def __init__(self, *_args: object, **_kwargs: object) -> None: + """Always raises ``TypeError``; use :meth:`bind`. + + Raises: + TypeError: every call. + """ + raise TypeError("use Activation.bind") + + @classmethod + def bind( + cls, + path: Path | str, + *, + store: ImmutableGitStore, + supported_capabilities: frozenset[str], + ) -> Activation: + """Only constructor; attach this instance to a pointer path. + + Does not write the file. A missing file becomes an in-memory + empty record (``current`` / ``previous`` / ``staged`` all + ``None``) and is not created. + + Args: + path: Pointer file path. + store: Published SHA store. Keyword-only; ``None`` is + refused. + supported_capabilities: Tokens this process can honor, + passed through to :func:`load_harness_catalog` on + ``stage``. Keyword-only; no default; ``None`` is + refused. + + Returns: + An :class:`Activation` whose properties match the file, or + all ``None`` when the file is missing. + + Raises: + TypeError: ``store`` or ``supported_capabilities`` is + ``None``. + ActivationVersionError: The file exists but is not a + version-1 pointer record. + """ + if store is None or supported_capabilities is None: + raise TypeError("store and supported_capabilities are required") + resolved = Path(path) + return cls._from_record( + resolved, + store=store, + supported_capabilities=supported_capabilities, + record=_load_record(resolved), + ) + + @classmethod + def _from_record( + cls, + path: Path, + *, + store: ImmutableGitStore, + supported_capabilities: frozenset[str], + record: _ActivationRecord, + ) -> Activation: + instance = object.__new__(cls) + instance._path = path + instance._store = store + instance._supported_capabilities = supported_capabilities + instance._record = record + return instance + + @property + def current(self) -> str | None: + """SHA currently activated, or ``None``.""" + return self._record.current + + @property + def previous(self) -> str | None: + """SHA that ``rollback`` would restore, or ``None``.""" + return self._record.previous + + @property + def staged(self) -> str | None: + """SHA waiting for ``promote``, or ``None``.""" + return self._record.staged + + def stage(self, sha: str) -> None: + """Mark ``sha`` as staged after catalog eligibility succeeds. + + Reloads the pointer from disk first. Loads + ``{tree_path(sha)}/harness.toml`` via :func:`load_harness_catalog` + (language gate, then every ``requires`` token ⊆ + ``supported_capabilities``). A new ``stage`` replaces any + already-staged SHA and leaves current/previous unchanged. + + Args: + sha: Commit SHA to stage. Must be a complete published tree. + + Raises: + IneligibleShaError: ``store.has(sha)`` is false, or + :func:`load_harness_catalog` raises + :class:`CatalogError`. The pointer file is not written. + """ + record = _load_record(self._path) + if not self._store.has(sha): + raise IneligibleShaError(sha) + try: + load_harness_catalog( + self._store.tree_path(sha), + sha, + self._supported_capabilities, + ) + except CatalogError as exc: + raise IneligibleShaError(sha) from exc + written = _ActivationRecord( + current=record.current, + previous=record.previous, + staged=sha, + ) + _write_record(self._path, written) + self._record = written + + def promote(self) -> None: + """Move staged to current; the old current becomes previous. + + Reloads the pointer from disk first. ``staged`` is cleared. + The previous previous is discarded; a second ``rollback`` then + has nothing to restore. + + Raises: + NothingStagedError: Reloaded ``staged`` is ``None``. + """ + record = _load_record(self._path) + if record.staged is None: + raise NothingStagedError("nothing staged") + written = _ActivationRecord( + current=record.staged, + previous=record.current, + staged=None, + ) + _write_record(self._path, written) + self._record = written + + def rollback(self) -> None: + """Restore previous as current and clear previous. + + Reloads the pointer from disk first. ``staged`` is unchanged. + + Raises: + NothingToRollbackError: Reloaded ``previous`` is ``None``. + """ + record = _load_record(self._path) + if record.previous is None: + raise NothingToRollbackError("nothing to rollback") + written = _ActivationRecord( + current=record.previous, + previous=None, + staged=record.staged, + ) + _write_record(self._path, written) + self._record = written diff --git a/src/molmcp/components/catalog.py b/src/molmcp/components/catalog.py new file mode 100644 index 0000000..6a4588f --- /dev/null +++ b/src/molmcp/components/catalog.py @@ -0,0 +1,466 @@ +"""Parse ``harness.toml``, then check eligibility. + +Load the TOML catalog at a checkout root (language gate), construct +:class:`HarnessCatalog`, then check eligibility against the caller's +``supported_capabilities`` and discard that set. Do not load +entrypoints or inspect git. +""" + +from __future__ import annotations + +import tomllib +from dataclasses import dataclass +from pathlib import Path + +from .models import ( + ALLOWED_REQUIRES, + SHA_PATTERN, + BundleSpec, + CatalogError, + ComponentKind, + ComponentSpec, +) + +_TOP_LEVEL_KEYS = frozenset({"requires", "component", "component_root"}) +_COMPONENT_KEYS = frozenset({"kind", "name", "path", "entrypoint"}) +_BUNDLE_KEYS = frozenset({"kind", "name", "members", "requires"}) + + +@dataclass(frozen=True, slots=True) +class ResolvedBundle: + """Bundle name plus member specs and the ordered ``requires`` union. + + Built by :meth:`HarnessCatalog.resolve_bundle`. Not stored on + :class:`HarnessCatalog`. ``requires`` is catalog-level tokens first, + then any bundle token not already seen (duplicates dropped, order + kept). + + Attributes: + name: Bundle name (``daily``, ``dev``, ...). + members: Member :class:`ComponentSpec` values, in catalog order. + requires: First-seen union of catalog then bundle tokens. + """ + + name: str + members: tuple[ComponentSpec, ...] + requires: tuple[str, ...] + + +@dataclass(frozen=True, slots=True) +class HarnessCatalog: + """Immutable catalog of components and bundles for one commit SHA. + + Identity is only ``sha`` (40-character lowercase git commit + fingerprint). Direct construction runs the language gate (valid SHA, + known ``requires`` tokens, unique ids). Bundles are optional: zero + bundles means the catalog is one implicit package of every + component. Eligibility against a runtime capability set is *not* a + field and is *not* checked here; only :func:`load_harness_catalog` + does that. + + Attributes: + sha: Caller-supplied 40-character lowercase hex git SHA. + requires: Catalog-level capability tokens (language-gate set). + components: Leaf :class:`ComponentSpec` rows (no bundles). + bundles: :class:`BundleSpec` rows (author-chosen names; may be + empty). + component_root: Tree-relative POSIX directory every component + ``path`` in this catalog resolves under, or ``""`` for the + tree itself. Declared last because the four fields above + carry no defaults. Component paths are carried beside it and + are never rewritten to include it. + + Raises: + CatalogError: Invalid SHA, unknown requires token, duplicate + id or bundle name, a bundle member id that is not in + ``components``, or a ``component_root`` that could escape + the tree. + """ + + sha: str + requires: tuple[str, ...] + components: tuple[ComponentSpec, ...] + bundles: tuple[BundleSpec, ...] + component_root: str = "" + + def __post_init__(self) -> None: + _validate_component_root(self.component_root) + if SHA_PATTERN.fullmatch(self.sha) is None: + raise CatalogError(f"invalid sha: {self.sha!r}") + for token in self.requires: + if token not in ALLOWED_REQUIRES: + raise CatalogError(f"unknown requires token: {token!r}") + names = tuple(bundle.name for bundle in self.bundles) + name_set = set(names) + ids = tuple(spec.id for spec in self.components) + if len(ids) != len(set(ids)): + raise CatalogError("duplicate component id") + if len(names) != len(name_set): + raise CatalogError("duplicate bundle name") + id_set = set(ids) + for bundle in self.bundles: + for member in bundle.members: + if member not in id_set: + raise CatalogError(f"unknown bundle member: {member!r}") + + def get(self, component_id: str) -> ComponentSpec: + """Return the component whose ``id`` is ``component_id``. + + Looks only at ``components``. Bundle names are not ids: + ``get("daily")`` fails even when a bundle named ``daily`` exists + (use :meth:`get_bundle`). + + Args: + component_id: Component id (``skill.daily``, not ``daily``). + + Returns: + The matching :class:`ComponentSpec`. + + Raises: + CatalogError: No component has that id. The message contains + ``unknown-id``. + """ + + for spec in self.components: + if spec.id == component_id: + return spec + raise CatalogError(f"unknown-id: {component_id!r}") + + def get_bundle(self, name: str) -> BundleSpec: + """Return the bundle named ``name``. + + Args: + name: Bundle name (``daily``, ``dev``, ...). + + Returns: + The matching :class:`BundleSpec`. + + Raises: + CatalogError: If no bundle has that name. The message + contains ``unknown-bundle``. + """ + + for bundle in self.bundles: + if bundle.name == name: + return bundle + raise CatalogError(f"unknown-bundle: {name!r}") + + def resolve_bundle(self, name: str) -> ResolvedBundle: + """Turn a bundle's member ids into specs and union ``requires``. + + ``requires`` is ``self.requires`` followed by that bundle's + tokens that have not already appeared, preserving first-seen + order. Eligibility is not checked again; a catalog that loaded + successfully already passed that gate. + + Args: + name: Bundle name to resolve. + + Returns: + A :class:`ResolvedBundle` (one-off view, not stored on this + catalog). + + Raises: + CatalogError: No bundle has that name (``unknown-bundle``). + """ + + bundle = self.get_bundle(name) + by_id = {spec.id: spec for spec in self.components} + members = tuple(by_id[member_id] for member_id in bundle.members) + requires = tuple(dict.fromkeys((*self.requires, *bundle.requires))) + return ResolvedBundle(name=bundle.name, members=members, requires=requires) + + def enabled_components( + self, names: tuple[str, ...] | None + ) -> tuple[ComponentSpec, ...]: + """Return the first-seen union of components selected by *names*. + + ``()`` is an explicit empty view. ``None`` means every bundle, + or every component when the catalog has no bundles. Unknown + names raise :class:`CatalogError` containing ``unknown-bundle``. + Components listed in more than one selected bundle appear once, + in enable-list then member order. + + Args: + names: Bundle names to include, ``None`` for all, or ``()`` + for none. + + Returns: + Selected :class:`ComponentSpec` rows. + + Raises: + CatalogError: A name is not a bundle in this catalog. The + message contains ``unknown-bundle``. + """ + if names == (): + return () + if not self.bundles: + if names is None: + return self.components + raise CatalogError("unknown-bundle: known: []") + selected = ( + names + if names is not None + else tuple(bundle.name for bundle in self.bundles) + ) + known = {bundle.name for bundle in self.bundles} + unknown = tuple(name for name in selected if name not in known) + if unknown: + listed = ", ".join(repr(name) for name in sorted(known)) + raise CatalogError(f"unknown-bundle: {unknown[0]!r}; known: [{listed}]") + kept: dict[str, ComponentSpec] = {} + for name in selected: + for spec in self.resolve_bundle(name).members: + kept.setdefault(spec.id, spec) + return tuple(kept.values()) + + +def load_harness_catalog( + tree: str | Path, + sha: str, + supported_capabilities: frozenset[str], +) -> HarnessCatalog: + """Load ``{tree}/harness.toml`` through the language gate, then eligibility. + + ``harness.toml`` is the TOML catalog at the checkout root. ``sha`` is + the caller's 40-character lowercase git commit SHA (Secure Hash + Algorithm fingerprint); it is stored as catalog identity and is not + read from the file. + + Two gates, in order: + + 1. Language — parse the file and construct :class:`HarnessCatalog`. + Unknown keys, unknown kinds, or a ``requires`` token outside + ``ALLOWED_REQUIRES`` fail here. ``ALLOWED_REQUIRES`` is not the + default for ``supported_capabilities`` and is not the eligibility + universe. + 2. Eligibility — every token in ``catalog.requires`` and in every + ``bundle.requires`` must be a subset of + ``supported_capabilities``. Then that set is discarded; it is + not stored on the catalog. An empty ``frozenset()`` is legal and + makes any non-empty ``requires`` ineligible. A token that is not + in ``ALLOWED_REQUIRES`` still fails the language gate even if it + appears in ``supported_capabilities`` (the message will not + contain ``ineligible``). + + This function does not import entrypoints, does not check that + component paths exist on disk, and does not talk to git. + + The optional ``component_root`` key is parsed here and carried onto + the catalog unchanged; it never moves ``harness.toml`` itself, which + always sits directly in ``tree``. This is also the only gate that can + see the key's *presence*, so it additionally refuses + ``component_root = ""``: :meth:`HarnessCatalog.__post_init__` + receives ``""`` from a defaulted field and from a written one alike + and cannot tell them apart. + + Args: + tree: Directory that contains ``harness.toml``. Named for + ``Checkout.tree``, which is what every caller passes; the + catalog's own ``component_root`` is a different directory and + is never spelled ``root`` here. + sha: 40-character lowercase hex git commit SHA. + supported_capabilities: Capability tokens this process can honor. + Required (no default). + + Returns: + A frozen :class:`HarnessCatalog` whose ``sha`` equals the + ``sha`` argument. + + Raises: + CatalogError: Missing file, invalid TOML, unknown field or kind, + language-gate failure, an empty ``component_root`` written + out, or an ineligible ``requires`` token (message contains + ``ineligible``). + TypeError: If ``supported_capabilities`` is omitted. + """ + + path = Path(tree) / "harness.toml" + if not path.is_file(): + raise CatalogError(f"missing harness.toml at {path}") + try: + parsed: object = tomllib.loads(path.read_text(encoding="utf-8")) + except tomllib.TOMLDecodeError as exc: + raise CatalogError(f"invalid harness.toml: {exc}") from exc + table = _as_table(parsed, "harness.toml") + _reject_unknown(table, _TOP_LEVEL_KEYS, "harness.toml") + requires = _require_str_tuple(table.get("requires", []), "requires") + component_root = "" + if "component_root" in table: + component_root = _require_string(table["component_root"], "component_root") + if not component_root: + raise CatalogError("component_root must not be empty when present") + components, bundles = _parse_component_rows(table.get("component", [])) + catalog = HarnessCatalog( + sha=sha, + requires=requires, + components=components, + bundles=bundles, + component_root=component_root, + ) + _assert_eligible(catalog, supported_capabilities) + return catalog + + +def _parse_component_rows( + raw_rows: object, +) -> tuple[tuple[ComponentSpec, ...], tuple[BundleSpec, ...]]: + if not isinstance(raw_rows, list): + raise CatalogError("component must be a list of tables") + components: list[ComponentSpec] = [] + bundles: list[BundleSpec] = [] + for raw_row in raw_rows: + parsed = _parse_row(_as_table(raw_row, "component")) + if isinstance(parsed, BundleSpec): + bundles.append(parsed) + else: + components.append(parsed) + return tuple(components), tuple(bundles) + + +def _parse_row(row: dict[str, object]) -> ComponentSpec | BundleSpec: + kind_value = row.get("kind") + if not isinstance(kind_value, str): + raise CatalogError("component row is missing kind") + # Wire kind "bundle" is not a ComponentKind; split before the enum. + if kind_value == "bundle": + return _parse_bundle_row(row) + return _parse_component_row(row, kind_value) + + +def _parse_bundle_row(row: dict[str, object]) -> BundleSpec: + _reject_unknown(row, _BUNDLE_KEYS, "bundle") + name = _require_string(row.get("name"), "bundle name") + members = _require_str_tuple(row.get("members"), "bundle members") + requires = _require_str_tuple(row.get("requires", []), "bundle requires") + return BundleSpec(name=name, members=members, requires=requires) + + +def _parse_component_row(row: dict[str, object], kind_value: str) -> ComponentSpec: + _reject_unknown(row, _COMPONENT_KEYS, "component") + try: + kind = ComponentKind(kind_value) + except ValueError as exc: + raise CatalogError(f"unknown component kind: {kind_value!r}") from exc + name = _require_string(row.get("name"), "component name") + path = _require_string(row.get("path"), "component path") + raw_entrypoint = row.get("entrypoint") + entrypoint = ( + None + if raw_entrypoint is None + else _require_string(raw_entrypoint, "entrypoint") + ) + return ComponentSpec( + kind=kind, + name=name, + id=f"{kind}.{name}", + path=path, + entrypoint=entrypoint, + ) + + +def _validate_component_root(value: str) -> None: + r"""Refuse a ``component_root`` that could escape the tree it joins onto. + + ``component_root`` is always the *first* component joined onto a + checkout tree, which is what makes each clause below load-bearing: + + * A backslash is not POSIX. + * Any ``:`` at all. ``"D:evil"`` carries no ``..``, holds no + backslash, and ``Path("D:evil").is_absolute()`` is ``False`` on + POSIX -- yet ``PureWindowsPath("C:/store/tree") / "D:evil"`` is + ``WindowsPath("D:evil")``: a drive on the first joined component + resets the anchor and discards the base. CI runs ``windows-latest``. + * Absolute, spelled as **two** clauses. + ``PureWindowsPath("/plugins").is_absolute()`` is ``False``, so + ``is_absolute()`` alone misses ``/plugins`` -- while + ``PureWindowsPath("C:/store/tree") / "/plugins"`` is + ``WindowsPath("C:/plugins")``. This is the same pair + ``_validate_component_path`` already carries, for the same reason. + * A ``".."`` **or ``"."``** segment, found by splitting on ``"/"`` + rather than reading ``PurePath.parts``, which silently drops ``.`` + and collapses ``//`` and would therefore miss both. Empty segments + are deliberately allowed: ``"plugins/mol/"`` and ``"plugins//mol"`` + both collapse to the same directory and escape nothing. + + **Path separators are deliberately NOT refused.** ``"plugins/mol"`` + is two segments and must stay legal -- that is the entire point of + the key, and the layout it exists to support. + ``ImmutableGitStore._sha_dir`` and ``harness.pointer_path`` refuse + separators because a SHA and a source name are single segments; this + is the opposite case, so restoring that check here would break the + only layout ``component_root`` was added for. + + ``""`` is legal: it means "the tree itself", and at construction time + a defaulted ``""`` and a written ``""`` are the same string. Refusing + the key *written* empty belongs to :func:`load_harness_catalog`, the + only gate that can still see presence. + + Args: + value: The catalog's ``component_root``, as authored. + + Raises: + CatalogError: The value could escape the tree. The message + carries ``value`` in ``repr`` form. + """ + + if "\\" in value: + raise CatalogError(f"component_root must be POSIX (no backslash): {value!r}") + if ":" in value: + raise CatalogError(f"component_root must not contain ':': {value!r}") + if Path(value).is_absolute() or value.startswith("/"): + raise CatalogError(f"component_root must be relative: {value!r}") + segments = value.split("/") + if ".." in segments or "." in segments: + raise CatalogError( + f"component_root must not contain '.' or '..' segments: {value!r}" + ) + + +def _assert_eligible( + catalog: HarnessCatalog, + supported_capabilities: frozenset[str], +) -> None: + needed = set(catalog.requires) + for bundle in catalog.bundles: + needed.update(bundle.requires) + unsupported = needed - supported_capabilities + if unsupported: + tokens = ", ".join(sorted(unsupported)) + raise CatalogError(f"ineligible requires: {tokens}") + + +def _reject_unknown( + data: dict[str, object], allowed: frozenset[str], where: str +) -> None: + unknown = sorted(set(data) - allowed) + if unknown: + raise CatalogError(f"unknown field(s) in {where}: {', '.join(unknown)}") + + +def _as_table(value: object, where: str) -> dict[str, object]: + if not isinstance(value, dict): + raise CatalogError(f"{where} must be a table") + table: dict[str, object] = {} + for key, item in value.items(): + if not isinstance(key, str): + raise CatalogError(f"{where} keys must be strings") + table[key] = item + return table + + +def _require_string(value: object, where: str) -> str: + if not isinstance(value, str): + raise CatalogError(f"{where} must be a string") + return value + + +def _require_str_tuple(value: object, where: str) -> tuple[str, ...]: + if not isinstance(value, list): + raise CatalogError(f"{where} must be a list of strings") + items: list[str] = [] + for item in value: + if not isinstance(item, str): + raise CatalogError(f"{where} must be a list of strings") + items.append(item) + return tuple(items) diff --git a/src/molmcp/components/git.py b/src/molmcp/components/git.py new file mode 100644 index 0000000..0edc0f5 --- /dev/null +++ b/src/molmcp/components/git.py @@ -0,0 +1,316 @@ +"""Stdlib git transports (GitHub HTTP, local checkout) and tarball extraction. + +Two implementations of one :class:`GitTransport` protocol. The GitHub one +reaches the network with ``urllib`` only; the caller supplies an optional +GitHub personal access token (PAT). The local one reaches no network at +all: it shells out to ``git`` inside a checkout already on disk. Neither +reads the environment. Request timeout is in seconds. Commit identity is +a SHA (Secure Hash Algorithm) hex digest. +""" + +from __future__ import annotations + +import io +import json +import subprocess +import tarfile +import urllib.error +import urllib.request +from pathlib import Path +from typing import Protocol + +_API = "https://api.github.com" +_CODELOAD = "https://codeload.github.com" +_TIMEOUT = 30 +_USER_AGENT = "molmcp" +_API_ACCEPT = "application/vnd.github+json" + +#: ``git rev-parse`` peel suffix: from any object, walk to the commit it +#: names. Load-bearing on an annotated tag, where a bare ``rev-parse`` +#: answers the *tag object's* SHA -- not a commit, and not something an +#: activation may be pinned to. +_TO_COMMIT = "^{commit}" + +#: Fallback first half of ``git archive --prefix``, used when the checkout +#: root has no directory name of its own (``/`` or a bare ``.``). +_ARCHIVE_PREFIX_FALLBACK = "harness" + + +class GitError(RuntimeError): + """Raised when a git remote request or archive extract fails.""" + + +class GitTransport(Protocol): + """Structural interface (``typing.Protocol``) for a git repository. + + Two primitives: resolve a *ref* (branch name, tag, or SHA) to a commit + SHA, and fetch that commit's gzip tarball. Combining them is the + caller's job. ``ref is None`` means resolve the repository default + branch first. Implementations raise :class:`GitError` on failure. + + ``owner`` and ``repo`` are the GitHub coordinate every implementation + is handed; one reading a checkout it was constructed with ignores + them. Which repository is spoken to is therefore the implementation's + own business, not something a caller can infer from the arguments. + """ + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + """Return the commit SHA (hex) for ``owner/repo`` at ``ref``. + + Args: + owner: Repository owner (user or org). + repo: Repository name. + ref: Branch, tag, or SHA. ``None`` selects the default branch. + + Returns: + Commit SHA as a hex digest. + + Raises: + GitError: Remote request failed, or the payload has no commit SHA. + """ + ... + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + """Return the gzip tarball bytes for ``owner/repo`` at ``sha``. + + Args: + owner: Repository owner (user or org). + repo: Repository name. + sha: Commit SHA (hex) to archive. + + Returns: + Raw ``tar.gz`` bytes. + + Raises: + GitError: Remote request failed. + """ + ... + + +class GitHubTransport: + """GitHub HTTP implementation of :class:`GitTransport`. + + Uses GitHub's JSON HTTP API (``api.github.com``) to resolve commits and + GitHub's archive host (``codeload.github.com``) to download a gzip + tarball. Timeout is :data:`_TIMEOUT` seconds on every request. + ``User-Agent`` is the literal ``molmcp``. Token is a personal access + token (PAT) or ``None`` (no ``Authorization`` header). + """ + + def __init__(self, token: str | None = None) -> None: + """Store an optional GitHub PAT. + + Args: + token: Personal access token, or ``None`` to send unauthenticated + requests. Not read from the environment. + """ + self._token = token + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + """Return the commit SHA (hex) for ``owner/repo`` at ``ref``. + + Args: + owner: Repository owner (user or org). + repo: Repository name. + ref: Branch, tag, or SHA. ``None`` looks up ``default_branch``, + then the commits URL; if that field is missing, uses + ``HEAD`` (git's name for the currently checked-out + revision). + + Returns: + Commit SHA as a hex digest. + + Raises: + GitError: HTTP/URL/OS failure, or JSON payload with no ``sha``. + """ + if ref is None: + info = self._get_json(f"{_API}/repos/{owner}/{repo}") + default = info.get("default_branch") + ref = default if isinstance(default, str) and default else "HEAD" + payload = self._get_json(f"{_API}/repos/{owner}/{repo}/commits/{ref}") + sha = payload.get("sha") + if not isinstance(sha, str) or not sha: + raise GitError(f"could not resolve {owner}/{repo}@{ref}") + return sha + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + """Return the gzip tarball bytes for ``owner/repo`` at ``sha``. + + Args: + owner: Repository owner (user or org). + repo: Repository name. + sha: Commit SHA (hex) to archive. + + Returns: + Raw ``tar.gz`` bytes from ``codeload.github.com``. + + Raises: + GitError: HTTP/URL/OS failure. + """ + url = f"{_CODELOAD}/{owner}/{repo}/tar.gz/{sha}" + return self._http_get(url, accept="application/octet-stream") + + def _get_json(self, url: str) -> dict[str, object]: + raw = self._http_get(url, accept=_API_ACCEPT) + try: + payload = json.loads(raw) + except (json.JSONDecodeError, ValueError) as exc: + raise GitError(f"GitHub request failed for {url}: {exc}") from exc + if not isinstance(payload, dict): + raise GitError(f"GitHub request failed for {url}: not a JSON object") + return payload + + def _http_get(self, url: str, *, accept: str) -> bytes: + headers = {"User-Agent": _USER_AGENT, "Accept": accept} + if self._token: + headers["Authorization"] = f"Bearer {self._token}" + request = urllib.request.Request(url, headers=headers) + try: + with urllib.request.urlopen(request, timeout=_TIMEOUT) as response: + return response.read() + except urllib.error.HTTPError as exc: + raise GitError(f"GitHub request failed ({exc.code}) for {url}") from exc + except (urllib.error.URLError, OSError) as exc: + raise GitError(f"GitHub request failed for {url}: {exc}") from exc + + +class LocalGitTransport: + """Local-checkout implementation of :class:`GitTransport`. + + A harness source may be a repository already on disk rather than a + GitHub coordinate: the way an operator serves a harness they are still + writing, and the only way to name one before it is published anywhere. + Both primitives shell out to ``git`` inside ``root``. Nothing here + opens a socket, and no token is involved. + + ``owner`` and ``repo`` are accepted because the protocol passes them, + and are ignored: the ``root`` this was constructed with is the whole + repository selection. Passing the coordinate of some other repository + does not reach that repository -- it reaches this checkout. + + :meth:`fetch_archive` archives the *committed tree* at a SHA, never the + working tree, which is what makes a local source pinned and rollbackable + in the same way a remote one is. Copying the directory instead would + make "pinned to a commit" mean "whatever the operator had unsaved when + we looked". + """ + + def __init__(self, root: Path) -> None: + """Store the checkout to read. + + Args: + root: Directory of a git repository. It is not validated here: + a root that is missing or is not a repository surfaces as + a :class:`GitError` from the first call, which is the same + failure a bad coordinate gets over HTTP. + """ + self._root = Path(root) + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + """Return the commit SHA (hex) that ``ref`` names in this checkout. + + The ref is peeled with :data:`_TO_COMMIT` before it is read. Without + that suffix an annotated tag -- how a harness release gets cut -- + resolves to the tag object's own SHA, which is not a commit and + names nothing ``git log`` can walk. + + Args: + owner: Ignored; see the class docstring. + repo: Ignored; see the class docstring. + ref: Branch, tag, or SHA. ``None`` selects the checked-out + revision, which is a local checkout's default branch. + + Returns: + Commit SHA as a hex digest. + + Raises: + GitError: ``git`` failed -- unknown ref, ``root`` missing or not + a repository, no ``git`` on PATH -- or answered nothing. + """ + target = "HEAD" if ref is None else ref + sha = self._git("rev-parse", "--verify", f"{target}{_TO_COMMIT}").strip() + if not sha: + raise GitError(f"could not resolve {target} in {self._root}") + return sha + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + """Return the gzip tarball bytes of the tree committed at ``sha``. + + Shaped like a GitHub commit archive: every member sits under one + top-level directory naming the commit, so + :func:`extract_git_archive` finds the same inner tree either + transport produced it. + + Args: + owner: Ignored; see the class docstring. + repo: Ignored; see the class docstring. + sha: Commit SHA (hex) to archive. + + Returns: + Raw ``tar.gz`` bytes of that commit's tree -- not of the working + tree, so an uncommitted edit is absent from it. + + Raises: + GitError: ``git`` failed -- unknown SHA, ``root`` missing or not + a repository, no ``git`` on PATH. + """ + prefix = f"{self._root.name or _ARCHIVE_PREFIX_FALLBACK}-{sha}/" + return self._git_bytes("archive", "--format=tar.gz", f"--prefix={prefix}", sha) + + def _git(self, *args: str) -> str: + return self._git_bytes(*args).decode("utf-8", "replace") + + def _git_bytes(self, *args: str) -> bytes: + """Run one git command inside ``root``; every failure is a GitError. + + A ``CalledProcessError`` must not escape: the protocol's contract is + :class:`GitError`, and a caller written against it would not catch + the subprocess type. ``git``'s own stderr is carried into the + message, since it is the only place the reason is written down. + """ + command = ["git", "-C", str(self._root), *args] + label = f"git {' '.join(args)} failed in {self._root}" + try: + completed = subprocess.run(command, check=True, capture_output=True) + except subprocess.CalledProcessError as exc: + detail = exc.stderr.decode("utf-8", "replace").strip() + raise GitError(f"{label}: {detail}") from exc + except OSError as exc: + raise GitError(f"{label}: {exc}") from exc + return completed.stdout + + +def extract_git_archive(data: bytes, dest: Path) -> Path: + """Extract a gzip git tarball and return the inner-tree root. + + A GitHub commit archive is a gzip-compressed tar whose members sit + under one top-level directory (for example ``owner-repo-sha/``). That + directory is the *inner tree* — the repository files — as opposed to + ``dest`` itself. + + Uses py3.12 ``filter="data"`` (blocks path traversal); older Python + falls back to unfiltered extract. + + Args: + data: Gzip-compressed tar bytes (a GitHub-style archive). + dest: Directory that should receive the extracted tree. + + Returns: + Path of the single top-level directory inside ``dest``. + + Raises: + GitError: Empty or corrupt archive, or no directory entry after + extract. + """ + try: + with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as tar: + try: + tar.extractall(dest, filter="data") + except TypeError: + tar.extractall(dest) + except (tarfile.TarError, OSError, EOFError, ValueError) as exc: + raise GitError(f"could not extract git archive: {exc}") from exc + subdirs = sorted(path for path in dest.iterdir() if path.is_dir()) + if not subdirs: + raise GitError("git archive contained no source directory") + return subdirs[0] diff --git a/src/molmcp/components/locator.py b/src/molmcp/components/locator.py new file mode 100644 index 0000000..ee76345 --- /dev/null +++ b/src/molmcp/components/locator.py @@ -0,0 +1,209 @@ +"""Parse a harness locator into one origin key. + +A locator is the string an operator writes: a GitHub URL or +``owner/repo[@ref]``, or a ``~/`` / absolute path. Spellings of the +same GitHub repository share one lowercase ``owner/repo`` origin key; +a *ref* is stored separately and is not identity. Local locators key +on the resolved path, which need not exist. + +Stdlib only — host and path are split by hand (no ``urllib``, no git). +Callers import :func:`parse_harness_locator` from this module; it is +not on :mod:`molmcp.components` ``__all__``. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import Literal + +_GITHUB_HOSTS = frozenset({"github.com", "www.github.com"}) +_HOST_PREFIXES = ("www.github.com/", "github.com/") + + +class LocatorError(ValueError): + """Raised when a harness locator cannot be accepted.""" + + +@dataclass(frozen=True, slots=True) +class ParsedHarnessLocator: + """One accepted harness locator, with identity separated from spelling. + + ``origin_key`` is the identity: lowercase ``owner/repo`` for GitHub + (``.git`` already stripped) or the resolved local path. ``ref`` is + empty when the locator did not name one. ``owner`` and ``repo`` are + empty for local locators. + + Attributes: + locator: Original text. + kind: ``"github"`` or ``"local"``. + origin_key: Canonical identity. + ref: Git ref, or ``""``. + owner: Lowercase GitHub owner, or ``""``. + repo: Lowercase GitHub repo without ``.git``, or ``""``. + """ + + locator: str + kind: Literal["github", "local"] + origin_key: str + ref: str + owner: str + repo: str + + +def parse_harness_locator(text: str) -> ParsedHarnessLocator: + """Parse one harness locator into a canonical origin. + + GitHub spellings (``https://github.com/Owner/repo``, + ``github.com/Owner/repo``, ``Owner/repo[@ref]``) share one lowercase + ``owner/repo`` origin key. ``.git`` and a trailing slash are stripped + from URL and host-prefixed forms. ``www.github.com`` is the same + host as ``github.com``. A *ref* after ``@`` on the shorthand form is + stored on the result and is not part of the origin key. + + Absolute paths (a leading ``/``, a Windows drive or UNC share) and + ``~/…`` (also ``~\\…`` on Windows) are local. The + origin key is ``str(Path(text).expanduser().resolve())``; the path + need not exist. A platform-absolute path may contain backslashes — + that is how ``Path`` stringifies on Windows. Relative paths, + whitespace, ``http://``, a ``github:`` prefix, SSH, extra URL path + segments, and backslashes *in a GitHub locator* raise. + + Args: + text: Locator as the operator wrote it. + + Returns: + Frozen parse result. ``locator`` is ``text`` unchanged. + + Raises: + LocatorError: If ``text`` is not an accepted locator. + """ + _reject_surface(text) + if _is_local_locator(text): + return ParsedHarnessLocator( + locator=text, + kind="local", + origin_key=str(Path(text).expanduser().resolve()), + ref="", + owner="", + repo="", + ) + if "\\" in text: + raise LocatorError(f"harness locator must be POSIX (no backslash): {text!r}") + owner, repo, ref = _parse_github(text) + return ParsedHarnessLocator( + locator=text, + kind="github", + origin_key=f"{owner}/{repo}", + ref=ref, + owner=owner, + repo=repo, + ) + + +def _reject_surface(text: str) -> None: + if not text: + raise LocatorError("harness locator must not be empty") + if any(ch.isspace() for ch in text): + raise LocatorError(f"harness locator must not contain whitespace: {text!r}") + if ( + text in {".", ".."} + or text.startswith("./") + or text.startswith("../") + or text.startswith(".\\") + or text.startswith("..\\") + ): + raise LocatorError(f"relative path is not a harness locator: {text!r}") + lowered = text.lower() + if lowered.startswith("http://"): + raise LocatorError(f"http:// is not a harness locator: {text!r}") + if lowered.startswith("github:"): + raise LocatorError(f"github: prefix is not a harness locator: {text!r}") + if lowered.startswith("ssh://") or lowered.startswith("git@"): + raise LocatorError(f"SSH is not a harness locator: {text!r}") + + +def _is_local_locator(text: str) -> bool: + """True when *text* names a filesystem path rather than a GitHub origin. + + ``~/…`` and a leading ``/`` are local on every platform — a + settings file that spells ``/opt/harness`` must not become a GitHub + shorthand just because Windows ``Path.is_absolute()`` is False + without a drive letter. ``Path.is_absolute()`` covers the rest: a + drive letter or UNC share on Windows. Existence is not required. + """ + if text.startswith(("~/", "~\\", "/")): + return True + return Path(text).is_absolute() + + +def _parse_github(text: str) -> tuple[str, str, str]: + lowered = text.lower() + if lowered.startswith("https://"): + owner, repo = _github_url_owner_repo(text, text[8:]) + return owner, repo, "" + for prefix in _HOST_PREFIXES: + if lowered.startswith(prefix): + owner, repo = _github_path_owner_repo(text, text[len(prefix) :]) + return owner, repo, "" + return _github_shorthand(text) + + +def _github_url_owner_repo(text: str, rest: str) -> tuple[str, str]: + if "/" not in rest: + raise LocatorError(f"invalid harness locator: {text!r}") + host, path = rest.split("/", 1) + if host.lower() not in _GITHUB_HOSTS: + raise LocatorError(f"invalid harness locator: {text!r}") + return _github_path_owner_repo(text, path) + + +def _github_path_owner_repo(text: str, path: str) -> tuple[str, str]: + if "?" in path or "#" in path or "@" in path: + raise LocatorError(f"invalid harness locator: {text!r}") + if path.endswith("/"): + path = path[:-1] + parts = path.split("/") + if len(parts) != 2 or not parts[0] or not parts[1]: + raise LocatorError(f"invalid harness locator: {text!r}") + return _normalize_owner_repo(text, parts[0], parts[1]) + + +def _github_shorthand(text: str) -> tuple[str, str, str]: + ref = "" + body = text + if "@" in text: + if text.count("@") != 1: + raise LocatorError(f"invalid harness locator: {text!r}") + body, ref = text.split("@", 1) + if not ref: + raise LocatorError(f"invalid harness locator: {text!r}") + if "?" in body or "#" in body or ":" in body: + raise LocatorError(f"invalid harness locator: {text!r}") + parts = body.split("/") + if len(parts) != 2 or not parts[0] or not parts[1]: + raise LocatorError(f"invalid harness locator: {text!r}") + owner, repo = _normalize_owner_repo(text, parts[0], parts[1]) + return owner, repo, ref + + +def _normalize_owner_repo(text: str, owner: str, repo: str) -> tuple[str, str]: + owner = owner.lower() + repo = repo.lower() + if repo.endswith(".git"): + repo = repo[:-4] + if not _is_github_owner(owner) or not _is_github_repo(repo): + raise LocatorError(f"invalid harness locator: {text!r}") + return owner, repo + + +def _is_github_owner(value: str) -> bool: + if not value or value[0] == "-" or value[-1] == "-": + return False + return all(ch.isalnum() or ch == "-" for ch in value) + + +def _is_github_repo(value: str) -> bool: + if not value or value in {".", ".."}: + return False + return all(ch.isalnum() or ch in "._-" for ch in value) diff --git a/src/molmcp/components/models.py b/src/molmcp/components/models.py new file mode 100644 index 0000000..513fa29 --- /dev/null +++ b/src/molmcp/components/models.py @@ -0,0 +1,204 @@ +"""Grammar for one component or bundle row in a harness catalog. + +This module owns the *language gate* for a single row: known kinds, +kebab-case names, POSIX paths, and ``requires`` tokens. It does not +parse TOML and does not decide *eligibility* (whether this process can +honor those tokens). ``ComponentSpec`` is one installable piece; +``BundleSpec`` is a named group of those pieces. ``ComponentKind`` has +no ``bundle`` member. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from enum import StrEnum +from pathlib import Path +from types import MappingProxyType + + +class CatalogError(ValueError): + """Raised when a harness catalog cannot be accepted. + + Three raisers, and the third is worth naming because it is the first + outside this package and outside a catalog object. Inside it: the + language gate (unknown key, unknown kind, token not in + ``ALLOWED_REQUIRES``, invalid SHA, and so on) and the eligibility + check (a grammatically valid ``requires`` token the caller cannot + honor). Eligibility failures are the ones whose message contains + ``ineligible``. + + Outside it: :class:`molmcp.harness.ComponentFold`, which folds several + catalogs into one served set. Its ``__post_init__`` raises this type + when its checkouts and their ``component_root`` strings disagree, and + its ``root_for`` raises it for a source the fold was not built from, + with ``unknown-source`` in the message — the same register + :meth:`HarnessCatalog.get`, :meth:`HarnessCatalog.get_bundle`, and + :meth:`HarnessCatalog.enabled_components` use. + So the type does not mean "one catalog file was rejected"; it means a + harness catalog, or something assembled directly out of several of + them, cannot be accepted. A second error family for that one message + was considered and refused: the register genuinely matches. + """ + + +class ComponentKind(StrEnum): + """Kind of one installable *component* (not a bundle). + + A component is a single piece declared in ``harness.toml``. A bundle + is a named group of those pieces and is a separate type + (:class:`BundleSpec`). ``ComponentKind("bundle")`` raises + ``ValueError``. + + Attributes: + SKILL: Instruction file an agent reads (path under ``skills/``). + AGENT: Agent definition file (path under ``agents/``). + RULE: Constraint file (path under ``rules/``). + PROVIDER: MCP provider module; requires an entrypoint. + OVERLAY: Discovery overlay module; requires an entrypoint. + """ + + SKILL = "skill" + AGENT = "agent" + RULE = "rule" + PROVIDER = "provider" + OVERLAY = "overlay" + + +#: Git commit SHA: 40 lowercase hexadecimal characters, nothing else. +#: SHA (Secure Hash Algorithm) here is the full commit fingerprint the +#: caller supplies as catalog identity. Uppercase hex, short SHAs, and +#: refs (``main``, tags) do not match. +SHA_PATTERN = re.compile(r"^[0-9a-f]{40}$") +#: Component and bundle names: kebab-case starting with a lowercase +#: letter (``daily``, ``molvis``). Owned here; not imported from the +#: provider SDK. +COMPONENT_NAME_PATTERN = re.compile(r"^[a-z][a-z0-9-]*$") +#: ``module:object`` string; this module never imports it. +_ENTRYPOINT_PATTERN = re.compile(r"^[A-Za-z_][A-Za-z0-9_.]*:[A-Za-z_][A-Za-z0-9_]*$") +_MEMBER_PATTERN = re.compile(r"^(skill|agent|rule|provider|overlay)\.[a-z][a-z0-9-]*$") +#: Tokens a ``requires`` list may mention (language gate only). +#: A token outside this set is invalid TOML, even if the caller put it +#: in ``supported_capabilities``. This set is not the default for that +#: argument and is not stored as an eligibility universe on the catalog. +ALLOWED_REQUIRES = frozenset({"provider-sdk", "harness-catalog"}) +#: POSIX directory prefix each ``ComponentKind`` path must start with. +#: The remainder after the prefix must be non-empty (``skills/`` alone +#: is rejected). +KIND_PATH_PREFIX = MappingProxyType( + { + ComponentKind.SKILL: "skills/", + ComponentKind.AGENT: "agents/", + ComponentKind.RULE: "rules/", + ComponentKind.PROVIDER: "providers/", + ComponentKind.OVERLAY: "overlays/", + } +) + +_ENTRYPOINT_KINDS = frozenset({ComponentKind.PROVIDER, ComponentKind.OVERLAY}) + + +@dataclass(frozen=True, slots=True) +class ComponentSpec: + """One installable component declared in a harness catalog. + + A component is a single piece (skill, agent, rule, provider, or + overlay). It is not a bundle. Construction rejects bad values; it + does not rewrite them. Frozen means the fields cannot change after + construction. + + An *entrypoint* is a ``module:object`` string (``pkg.mod:Class``) + naming a Python object to import later. It is required for + ``provider`` and ``overlay``, and must be ``None`` for every other + kind. This class never imports that string. + + Attributes: + kind: One :class:`ComponentKind` value (never bundle). + name: Kebab-case name matching ``COMPONENT_NAME_PATTERN``. + id: Must equal ``f"{kind}.{name}"`` (TOML has no ``id`` key). + path: Relative POSIX path under that kind's ``KIND_PATH_PREFIX``, + with no backslash, no ``..`` segment, and at least one + character after the prefix. + entrypoint: ``module:object`` string, or ``None``. + + Raises: + CatalogError: If any field fails the grammar above. + """ + + kind: ComponentKind + name: str + id: str + path: str + entrypoint: str | None = None + + def __post_init__(self) -> None: + if not isinstance(self.kind, ComponentKind): + raise CatalogError("kind must be a ComponentKind") + if COMPONENT_NAME_PATTERN.fullmatch(self.name) is None: + raise CatalogError(f"invalid component name: {self.name!r}") + expected_id = f"{self.kind}.{self.name}" + if self.id != expected_id: + raise CatalogError(f"id must be {expected_id!r}, got {self.id!r}") + _validate_component_path(self.kind, self.path) + if self.kind in _ENTRYPOINT_KINDS: + if ( + not isinstance(self.entrypoint, str) + or _ENTRYPOINT_PATTERN.fullmatch(self.entrypoint) is None + ): + raise CatalogError( + f"{self.kind} entrypoint must be a module:object string" + ) + elif self.entrypoint is not None: + raise CatalogError(f"{self.kind} entrypoint must be None") + + +@dataclass(frozen=True, slots=True) +class BundleSpec: + """Named grouping of component ids, with optional ``requires`` tokens. + + A bundle is a preset such as ``daily`` or ``dev``. It is not a + :class:`ComponentKind` and cannot appear as a member of another + bundle. ``requires`` lists capability tokens the file is allowed to + name (language gate). Whether this process can honor them is + eligibility, checked later by ``load_harness_catalog``. + + Attributes: + name: Kebab-case bundle name (same pattern as component names). + members: Non-empty tuple of ``kind.name`` ids + (``skill.daily``, never ``bundle.daily``). + requires: Tokens, each of which must be in ``ALLOWED_REQUIRES``. + + Raises: + CatalogError: Empty members, malformed member id, unknown + requires token, or invalid name. + """ + + name: str + members: tuple[str, ...] + requires: tuple[str, ...] = () + + def __post_init__(self) -> None: + if COMPONENT_NAME_PATTERN.fullmatch(self.name) is None: + raise CatalogError(f"invalid bundle name: {self.name!r}") + if not self.members: + raise CatalogError("bundle members must not be empty") + for member in self.members: + if _MEMBER_PATTERN.fullmatch(member) is None: + raise CatalogError(f"invalid bundle member: {member!r}") + for token in self.requires: + if token not in ALLOWED_REQUIRES: + raise CatalogError(f"unknown requires token: {token!r}") + + +def _validate_component_path(kind: ComponentKind, path: str) -> None: + if not path: + raise CatalogError("path must not be empty") + if "\\" in path: + raise CatalogError("path must be POSIX (no backslash)") + if Path(path).is_absolute() or path.startswith("/"): + raise CatalogError("path must be relative") + if ".." in path.split("/"): + raise CatalogError("path must not contain '..' segments") + prefix = KIND_PATH_PREFIX[kind] + if not path.startswith(prefix) or len(path) <= len(prefix): + raise CatalogError(f"path must start with {prefix!r} and continue") diff --git a/src/molmcp/components/store.py b/src/molmcp/components/store.py new file mode 100644 index 0000000..636b75a --- /dev/null +++ b/src/molmcp/components/store.py @@ -0,0 +1,188 @@ +"""Immutable SHA-keyed store of published git archives. + +``publish`` is the only write path. Each SHA is one directory under +``/commits//``, swapped in with a single ``os.replace`` of +the whole directory (``metadata.json`` plus flattened ``tree/``). +Incomplete directories are not hits. The store does not write pointer +files and does not create ``refs/`` or ``pointers/``. +""" + +from __future__ import annotations + +import json +import os +import shutil +import tempfile +from pathlib import Path + +from .git import GitTransport, extract_git_archive + +_RESERVED_SHA_KEYS = frozenset({".", "..", "refs", "pointers", "hints"}) + + +class StoreError(Exception): + """Base error for :class:`ImmutableGitStore` operations.""" + + +class UnknownShaError(StoreError): + """Raised when a SHA has no complete published directory. + + Complete means ``metadata.json`` is a file and ``tree/`` is a + directory. Missing or incomplete SHAs fail at the store boundary. + """ + + +class ShaConflictError(StoreError): + """Raised when ``publish`` would change a SHA's owner or repo. + + Provenance is the ``owner`` and ``repo`` kwargs stored in + ``metadata.json``. The existing tree is left unchanged. + """ + + +class ImmutableGitStore: + """Disk store that publishes one complete SHA directory at a time. + + A *SHA directory* is ``/commits//`` containing + ``metadata.json`` (provenance ``owner`` / ``repo``) and ``tree/`` + (the catalog root). The *inner tree* is the single top-level + directory inside the commit tarball returned by + :func:`extract_git_archive`. *Flatten* means relocating that + directory to ``tree/`` so ``harness.toml`` is a direct child of + :meth:`tree_path` and ``tree/-/`` does not exist. + + Construct with :meth:`__init__`. ``publish`` fetches, flattens, and + ``os.replace``s the whole SHA directory. + + Args: + root: Store directory. Created as needed when publishing. + transport: :class:`GitTransport` supplying ``fetch_archive``. + + Raises: + TypeError: ``root`` or ``transport`` is ``None``. + """ + + def __init__(self, root: Path | str, transport: GitTransport) -> None: + """Store ``root`` as a :class:`~pathlib.Path` and the transport. + + Args: + root: Store directory (string or path). + transport: Git archive transport. + + Raises: + TypeError: ``root`` or ``transport`` is ``None``. + """ + if root is None or transport is None: + raise TypeError("root and transport are required") + self._root = Path(root) + self._transport = transport + + def has(self, sha: str) -> bool: + """Return whether ``sha`` has a complete published directory. + + Args: + sha: Commit SHA used as the directory key. + + Returns: + ``True`` only when ``metadata.json`` is a file and ``tree/`` + is a directory under ``commits//``. + """ + sha_dir = self._sha_dir(sha) + return (sha_dir / "metadata.json").is_file() and (sha_dir / "tree").is_dir() + + def tree_path(self, sha: str) -> Path: + """Return the flattened catalog root for a complete SHA. + + Args: + sha: Commit SHA used as the directory key. + + Returns: + Path of ``commits//tree/``. + + Raises: + UnknownShaError: ``sha`` is missing or incomplete. + """ + if not self.has(sha): + raise UnknownShaError(sha) + return self._sha_dir(sha) / "tree" + + def publish(self, sha: str, *, owner: str, repo: str) -> Path: + """Fetch, flatten, and atomically install ``sha`` if needed. + + A complete directory with the same ``owner`` and ``repo`` is a + no-op (no fetch, no tree replace). A complete directory with a + different provenance raises :class:`ShaConflictError` and does + not replace the tree. Missing or incomplete directories are + fetched, assembled in a temp directory under ``commits/``, and + installed with one ``os.replace`` of the whole SHA directory. + + Args: + sha: Commit SHA to publish (directory key). + owner: Provenance owner written to ``metadata.json``. + repo: Provenance repository written to ``metadata.json``. + + Returns: + Catalog root (``tree/``) for ``sha``. + + Raises: + ShaConflictError: ``sha`` is already published under a + different owner or repo. + StoreError: ``sha`` is not a usable directory key. + """ + if self.has(sha): + payload = self._read_metadata(sha) + if payload.get("owner") == owner and payload.get("repo") == repo: + return self.tree_path(sha) + raise ShaConflictError( + f"{sha} already published as " + f"{payload.get('owner')}/{payload.get('repo')}" + ) + + commits = self._root / "commits" + commits.mkdir(parents=True, exist_ok=True) + dest = self._sha_dir(sha) + tmp = Path(tempfile.mkdtemp(prefix=".tmp-", dir=commits)) + try: + staging = tmp / "sha" + unpack = tmp / "unpack" + staging.mkdir() + unpack.mkdir() + data = self._transport.fetch_archive(owner, repo, sha) + inner = extract_git_archive(data, unpack) + os.replace(inner, staging / "tree") + (staging / "metadata.json").write_text( + json.dumps({"sha": sha, "owner": owner, "repo": repo}), + encoding="utf-8", + ) + if dest.exists(): + aside = Path(tempfile.mkdtemp(prefix=".tmp-old-", dir=commits)) + try: + os.replace(dest, aside / "sha") + os.replace(staging, dest) + finally: + shutil.rmtree(aside, ignore_errors=True) + else: + os.replace(staging, dest) + finally: + shutil.rmtree(tmp, ignore_errors=True) + return self.tree_path(sha) + + def _sha_dir(self, sha: str) -> Path: + if ( + not sha + or sha in _RESERVED_SHA_KEYS + or Path(sha).is_absolute() + or os.sep in sha + or "/" in sha + or "\\" in sha + or (os.altsep is not None and os.altsep in sha) + ): + raise StoreError(f"invalid sha {sha!r}") + return self._root / "commits" / sha + + def _read_metadata(self, sha: str) -> dict[str, object]: + path = self._sha_dir(sha) / "metadata.json" + payload = json.loads(path.read_text(encoding="utf-8")) + if not isinstance(payload, dict): + raise StoreError(f"metadata for {sha} is not an object") + return payload diff --git a/src/molmcp/discovery/source/github.py b/src/molmcp/discovery/source/github.py index c37ea23..ea4d01c 100644 --- a/src/molmcp/discovery/source/github.py +++ b/src/molmcp/discovery/source/github.py @@ -1,33 +1,40 @@ """GitHub source resolution. -Resolves a ``github:owner/repo[@ref]`` spec to an immutable snapshot by -resolving the ref to a commit SHA (the snapshot id) and downloading that -commit's tarball. All network access is stdlib ``urllib`` and is -confined to this module, so a restricted-network deployment can disable -GitHub and keep local discovery fully working. +Turn a ``github:owner/repo[@ref]`` spec into an immutable snapshot. +A *ref* is a branch name, tag, or SHA (Secure Hash Algorithm hex +digest, the git commit id). The snapshot id is ``github:commit:``, +not the SHA alone. + +Network access is a :class:`~molmcp.components.git.GitTransport` +(default :class:`~molmcp.components.git.GitHubTransport`), built only +by :func:`_transport` from ``config.github_token`` (optional GitHub +personal access token, PAT). This module parses the spec, extracts the +*inner tree* (the single top-level directory inside the commit tarball) +into ``SnapshotCache.raw_dir`` (``/snapshots//raw/``), +records that path in a ``.extracted`` marker, and maps +:class:`~molmcp.components.git.GitError` to :class:`SourceError`. """ from __future__ import annotations -import io -import json import shutil -import tarfile import time -import urllib.error -import urllib.request from pathlib import Path +from molmcp.components.git import ( + GitError, + GitHubTransport, + GitTransport, + extract_git_archive, +) + from ..config import DiscoveryConfig from .resolver import Snapshot, SnapshotId, SourceError from .walk import walk_files -_API = "https://api.github.com" -_CODELOAD = "https://codeload.github.com" -_TIMEOUT = 30 - def _parse_github_spec(spec: str) -> tuple[str, str, str | None]: + """Parse ``github:owner/repo[@ref]`` into ``(owner, repo, ref)``.""" body = spec[len("github:") :] if spec.startswith("github:") else spec ref: str | None = None if "@" in body: @@ -40,68 +47,50 @@ def _parse_github_spec(spec: str) -> tuple[str, str, str | None]: return parts[0], parts[1], (ref or None) -def _http_get( - url: str, - token: str | None = None, - accept: str = "application/vnd.github+json", -) -> bytes: - headers = {"User-Agent": "molmcp-discovery", "Accept": accept} - if token: - headers["Authorization"] = f"Bearer {token}" - request = urllib.request.Request(url, headers=headers) - try: - with urllib.request.urlopen(request, timeout=_TIMEOUT) as response: - return response.read() - except urllib.error.HTTPError as exc: - raise SourceError(f"GitHub request failed ({exc.code}) for {url}") from exc - except (urllib.error.URLError, OSError, ValueError) as exc: - raise SourceError(f"GitHub request failed for {url}: {exc}") from exc - - -def resolve_ref(owner: str, repo: str, ref: str | None, config: DiscoveryConfig) -> str: - """Resolve a branch/tag/ref (or the default branch) to a commit SHA.""" - token = config.github_token - if ref is None: - info = json.loads(_http_get(f"{_API}/repos/{owner}/{repo}", token)) - ref = info.get("default_branch", "HEAD") - payload = json.loads(_http_get(f"{_API}/repos/{owner}/{repo}/commits/{ref}", token)) - sha = payload.get("sha") - if not sha: - raise SourceError(f"could not resolve {owner}/{repo}@{ref}") - return sha +def _transport(config: DiscoveryConfig) -> GitTransport: + """Build the :class:`GitTransport` for this config (PAT from ``github_token``).""" + return GitHubTransport(token=config.github_token) def latest_commit(spec: str, config: DiscoveryConfig) -> str: - """Return the current commit SHA a github spec points at.""" + """Return the current commit SHA a ``github:`` spec points at. + + A SHA (Secure Hash Algorithm) here is the hex digest GitHub uses as a + commit id. This call only resolves the ref; it does not download the + archive. Network access goes through :func:`_transport`. + + Args: + spec: ``github:owner/repo[@ref]``. A *ref* is a branch name, tag, or + SHA; omitted means the repository default branch. + config: Discovery settings. ``github_token`` is the optional GitHub + personal access token (PAT) handed to the transport. + + Returns: + Commit SHA as a hex digest. + + Raises: + SourceError: Invalid spec, or a :class:`~molmcp.components.git.GitError` + from the transport (mapped with ``raise SourceError(str(exc)) + from exc``). + """ owner, repo, ref = _parse_github_spec(spec) - return resolve_ref(owner, repo, ref, config) - - -def _safe_extract(tar: tarfile.TarFile, dest: Path) -> None: + transport = _transport(config) try: - tar.extractall(dest, filter="data") # py3.12+: blocks traversal - except TypeError: # pragma: no cover - older Python - tar.extractall(dest) - - -def _download_source( - owner: str, repo: str, sha: str, raw_dir: Path, config: DiscoveryConfig -) -> Path: - raw_dir.mkdir(parents=True, exist_ok=True) - url = f"{_CODELOAD}/{owner}/{repo}/tar.gz/{sha}" - data = _http_get(url, config.github_token, accept="application/octet-stream") - with tarfile.open(fileobj=io.BytesIO(data), mode="r:gz") as tar: - _safe_extract(tar, raw_dir) - subdirs = sorted(d for d in raw_dir.iterdir() if d.is_dir()) - if not subdirs: - raise SourceError("GitHub tarball contained no source directory") - return subdirs[0] + return transport.resolve_commit(owner, repo, ref) + except GitError as exc: + raise SourceError(str(exc)) from exc def _ensure_source( - owner: str, repo: str, sha: str, raw_dir: Path, config: DiscoveryConfig + owner: str, repo: str, sha: str, raw_dir: Path, transport: GitTransport ) -> Path: - """Return the extracted source root, downloading it once if needed.""" + """Return the inner-tree root, fetching the archive only when needed. + + ``raw_dir / ".extracted"`` is a marker file holding the inner-tree + absolute path. If that file exists and the path is a directory, return + it. Otherwise delete ``raw_dir``, download via ``transport.fetch_archive``, + extract with :func:`extract_git_archive`, and rewrite the marker. + """ marker = raw_dir / ".extracted" if marker.is_file(): root = Path(marker.read_text(encoding="utf-8").strip()) @@ -109,20 +98,52 @@ def _ensure_source( return root if raw_dir.exists(): shutil.rmtree(raw_dir, ignore_errors=True) - root = _download_source(owner, repo, sha, raw_dir, config) + raw_dir.mkdir(parents=True, exist_ok=True) + data = transport.fetch_archive(owner, repo, sha) + root = extract_git_archive(data, raw_dir) marker.write_text(str(root), encoding="utf-8") return root def resolve_github(spec: str, config: DiscoveryConfig) -> Snapshot: - """Resolve a ``github:owner/repo[@ref]`` spec to a snapshot.""" + """Resolve a ``github:owner/repo[@ref]`` spec to an immutable snapshot. + + Resolves the ref to a commit SHA via :func:`_transport`, then places + that commit's files under ``SnapshotCache.raw_dir`` — the per-snapshot + directory ``/snapshots//raw/``. GitHub tarballs wrap + the repo in one top-level folder (for example ``owner-repo-sha/``); + that folder is the *inner tree* and becomes ``Snapshot.root_dir``, not + ``raw/`` itself. A ``.extracted`` marker file inside ``raw/`` stores + the inner-tree absolute path so a later call can skip the download. + + Args: + spec: ``github:owner/repo[@ref]``. A *ref* is a branch name, tag, or + SHA; omitted means the repository default branch. + config: Discovery settings, including ``cache_dir`` and optional + ``github_token`` (GitHub PAT). + + Returns: + Snapshot whose ``snapshot_id`` is ``github:commit:``, + ``commit`` is that SHA, and ``root_dir`` is the inner tree. + + Raises: + SourceError: Invalid spec, or a :class:`~molmcp.components.git.GitError` + from resolve/fetch/extract (mapped with ``from exc``). + """ from ..cache.snapshotcache import SnapshotCache owner, repo, ref = _parse_github_spec(spec) - sha = resolve_ref(owner, repo, ref, config) + transport = _transport(config) + try: + sha = transport.resolve_commit(owner, repo, ref) + except GitError as exc: + raise SourceError(str(exc)) from exc snapshot_id = SnapshotId("github", "commit", sha) raw_dir = SnapshotCache(config).raw_dir(str(snapshot_id)) - root = _ensure_source(owner, repo, sha, raw_dir, config) + try: + root = _ensure_source(owner, repo, sha, raw_dir, transport) + except GitError as exc: + raise SourceError(str(exc)) from exc files = tuple(walk_files(root, config)) return Snapshot( snapshot_id=str(snapshot_id), diff --git a/src/molmcp/evolution/__init__.py b/src/molmcp/evolution/__init__.py new file mode 100644 index 0000000..85693c7 --- /dev/null +++ b/src/molmcp/evolution/__init__.py @@ -0,0 +1,79 @@ +"""The gate that says whether a harness change is worth taking. + +A *harness* is the set of skills, rules and agent definitions that shape +how an agent works. Changing one is cheap; knowing whether the change +helped is not. :mod:`~molmcp.evolution.evaluate` is the part that can be +made reproducible: given a challenger checkout, the champion's sha, and +readings taken on both, it returns one frozen +:class:`~molmcp.evolution.evaluate.EvaluationReport` — accepted or not, +and the single reason why. + +Four readings, compared one at a time and never summed into a score. A +composite would let a cheap win pay for a broken run, so the report names +which reading decided rather than hiding it in an average. The comparison +runs on the un-rounded means and only the report rounds: a champion +averaging 10.0 against a challenger averaging 10.4 both round to 10, and +rounding first would let that regression through as a tie. + +The two seams it needs — :class:`~molmcp.evolution.evaluate.ContractRunner` +and :class:`~molmcp.evolution.evaluate.ReplayFn` — are keyword-only with +no default at all, because any default would have to be a real host and a +host is exactly what this module is kept away from. Who fills them, and +how the readings are taken, is the caller's problem; this module only +decides. + +It moves no pointer. The report is a verdict, and acting on one belongs +to whoever holds the pointer. + +Leaf package, a sibling of :mod:`molmcp.helpers`: the standard library +and its own types. It does not import FastMCP, does not read settings, +and is not re-exported from :mod:`molmcp` — a verdict is not an MCP tool. +""" + +from .evaluate import ( + ACCEPTED, + DEFAULT_SEEDS, + DROP_CALL_COUNT, + DROP_LATENCY_S, + DROP_TOKENS, + DROP_TOOL_ERRORS, + NO_PRACTICAL_GAIN, + REGRESSION_FAILED, + WORSE_CALL_COUNT, + WORSE_LATENCY, + WORSE_TOKENS, + WORSE_TOOL_ERRORS, + Challenger, + ContractOutcome, + ContractRunner, + EvalCase, + EvaluationError, + EvaluationReport, + Metrics, + ReplayFn, + evaluate, +) + +__all__ = [ + "ACCEPTED", + "DEFAULT_SEEDS", + "DROP_CALL_COUNT", + "DROP_LATENCY_S", + "DROP_TOKENS", + "DROP_TOOL_ERRORS", + "NO_PRACTICAL_GAIN", + "REGRESSION_FAILED", + "WORSE_CALL_COUNT", + "WORSE_LATENCY", + "WORSE_TOKENS", + "WORSE_TOOL_ERRORS", + "Challenger", + "ContractOutcome", + "ContractRunner", + "EvalCase", + "EvaluationError", + "EvaluationReport", + "Metrics", + "ReplayFn", + "evaluate", +] diff --git a/src/molmcp/evolution/evaluate.py b/src/molmcp/evolution/evaluate.py new file mode 100644 index 0000000..7302389 --- /dev/null +++ b/src/molmcp/evolution/evaluate.py @@ -0,0 +1,415 @@ +"""Held-out gate: four readings compared one by one, never summed. + +A *challenger* is a checkout of the harness that wants to replace the +current *champion*. :func:`evaluate` decides whether it may and says why, +in one frozen :class:`EvaluationReport`: the graduated regression cases +must still pass, and the held-out cases are then replayed on both sides +under the frozen seeds :data:`DEFAULT_SEEDS` and compared on four +readings — ``tool_errors``, ``call_count``, ``tokens`` and ``latency_s`` +(seconds). All four are lower-is-better, and each is compared on its own +against its own ``DROP_*`` threshold. There is no weighted total, so a +gain in one reading can never buy a regression in another, and the +verdict is a literal reason rather than a number. + +Two disciplines are easy to lose: + +* *The float mean decides; the rounded mean is only reported.* Three of + the four readings are ints and the report stores ``round(mean)``, but + the worse/better comparison runs on the un-rounded seed mean. Rounding + first would let a real regression of a third of a call per seed vanish + into a tie — and let a real gain of the same size vanish with it. +* *The report is a verdict, not a promotion.* It carries no pointer, no + stage and no score, and :func:`evaluate` reads and writes no + active/previous pointer and no session lock. Moving the champion + pointer belongs to a later leaf; this one only says accepted or not, + and why. + +Leaf module: the standard library and its own types. It imports no +FastMCP, no MCP and nothing +from the runtime that composes planes — which is why the two seams it +needs, :class:`ContractRunner` and :class:`ReplayFn`, are keyword-only +parameters with no default at all. A default would have to be a real +host, and the host is exactly what this module is kept away from. Tests +inject fakes; the production pair is injected by the CI gate. +""" + +from __future__ import annotations + +from collections.abc import Sequence +from dataclasses import dataclass +from pathlib import Path +from typing import Protocol + +#: The challenger replaces the champion. The only reason an accepted +#: report may carry, and the only one a rejection may not. +ACCEPTED = "accepted" + +#: A graduated regression case failed on the challenger tree. Decided +#: before any held-out replay, so it says nothing about the readings. +REGRESSION_FAILED = "regression_failed" + +#: The challenger made more tool errors than the champion. +WORSE_TOOL_ERRORS = "worse_tool_errors" + +#: The challenger needed more tool calls than the champion. +WORSE_CALL_COUNT = "worse_call_count" + +#: The challenger spent more tokens than the champion. +WORSE_TOKENS = "worse_tokens" + +#: The challenger took longer than the champion. +WORSE_LATENCY = "worse_latency" + +#: Nothing got worse, but nothing got better either. A tie is a +#: rejection: the champion keeps the seat it already holds. +NO_PRACTICAL_GAIN = "no_practical_gain" + +#: The seeds every held-out replay uses unless a caller names its own. +#: Frozen at three so two runs of the same pair of trees compare the same +#: way; a report always records the seeds it actually used. +DEFAULT_SEEDS: tuple[int, ...] = (1, 2, 3) + +#: How much worse than the champion each reading may get before it is a +#: regression, in that reading's own unit (errors, calls, tokens, +#: seconds). All zero: there is no noise band here, because the replay is +#: seeded and any move in the wrong direction is a real one. +DROP_TOOL_ERRORS = 0 +DROP_CALL_COUNT = 0 +DROP_TOKENS = 0 +DROP_LATENCY_S = 0.0 + + +class EvaluationError(ValueError): + """A gate that cannot be run, or a verdict that cannot be held. + + Raised for an empty seed list, an empty held-out suite, and any + :class:`EvaluationReport` whose ``accepted`` and ``reason`` disagree. + A ``ValueError`` because every case is a bad value handed to this + layer, not a failure of something it called. + """ + + +@dataclass(frozen=True, slots=True) +class EvalCase: + """One case a runner or a replay is asked to work through. + + Identity only. What the case *does* lives with whoever executes it — + this module hands cases across a seam and never reads inside one. + + Attributes: + id: Stable identity of the case, as the executing side knows it. + """ + + id: str + + +@dataclass(frozen=True, slots=True) +class Metrics: + """One side's readings from a held-out replay. + + Exactly four numbers, all lower-is-better, all compared + independently. There is deliberately no score, total or rank here: a + single number would let a token saving pay for an extra tool error, + and this gate does not make that trade. + + Attributes: + tool_errors: Count of tool calls that returned an error. + call_count: Count of tool calls made. + tokens: Count of tokens spent. + latency_s: Wall-clock duration in **seconds**. The only float of + the four, and the only reading the report keeps un-rounded. + """ + + tool_errors: int + call_count: int + tokens: int + latency_s: float + + +#: What both sides read when the regression contract short-circuits: the +#: replay never ran, so there is nothing to report but zeroes. +_ZERO_METRICS = Metrics(tool_errors=0, call_count=0, tokens=0, latency_s=0.0) + + +@dataclass(frozen=True, slots=True) +class ContractOutcome: + """What a :class:`ContractRunner` says about the graduated suite. + + Attributes: + passed: Whether every case given to the runner passed. + failed_case_ids: Ids of the cases that did not, empty when + ``passed``. Carried for the report's readers, not read here: + one failure is already the whole verdict. + """ + + passed: bool + failed_case_ids: tuple[str, ...] + + +class Challenger(Protocol): + """The checkout under evaluation, read for three names only. + + Duck-typed on purpose, and named for what it is: a tree someone has + already built, not a patch someone has proposed. The report field is + still ``candidate_sha``. + + :func:`evaluate` reads ``sha`` and nothing else; ``component`` and + ``affected_paths`` are here because callers pass one object around, + and this module neither interprets a path nor checks one against a + whitelist. + """ + + @property + def sha(self) -> str: + """Full commit sha of the challenger checkout.""" + + @property + def component(self) -> str: + """Id of the harness component the challenger changes.""" + + @property + def affected_paths(self) -> Sequence[str]: + """Repository paths the challenger touches.""" + + +class ContractRunner(Protocol): + """Runs the graduated regression cases against a checked-out tree. + + The production runner starts a real host; this module only ever + holds the seam, which is why it has no default. + """ + + def run(self, tree: Path, cases: Sequence[EvalCase]) -> ContractOutcome: + """Run *cases* against the tree at *tree* and report the outcome.""" + ... + + +class ReplayFn(Protocol): + """Replays the held-out cases against one side under one seed. + + The two sides are named differently on purpose: the champion by its + full sha, because resolving it to a tree belongs behind this seam, + and the challenger by the tree a caller already checked out. + """ + + def __call__( + self, target: str | Path, cases: Sequence[EvalCase], seed: int + ) -> Metrics: + """Replay *cases* against *target* under *seed* and read the metrics.""" + ... + + +@dataclass(frozen=True, slots=True) +class EvaluationReport: + """The whole verdict on one challenger, and the only home for it. + + A verdict, not a promotion: there is no pointer, stage or score field + here, and constructing one whose ``accepted`` disagrees with its + ``reason`` or with ``regression_passed`` raises rather than + producing a report a later reader would have to second-guess. + + Attributes: + accepted: Whether the challenger may replace the champion. True + only together with ``reason == ACCEPTED`` and + ``regression_passed``. + reason: One of the seven frozen literals in this module. + candidate_sha: Full sha of the challenger that was evaluated. + champion_sha: Full sha of the champion it was compared against. + seeds: The seeds the replay actually used, in call order. + regression_passed: Whether the graduated suite still passed. True + when there were no graduated cases to run. + champion_metrics: The champion's seed means — the three counts + rounded to ints, ``latency_s`` the mean in seconds. Zeroed + when the regression contract short-circuited the replay. + challenger_metrics: The challenger's seed means, same shape. + + Raises: + EvaluationError: ``accepted`` is True while ``regression_passed`` + is False, ``accepted`` is True under any reason other than + ``ACCEPTED``, or ``accepted`` is False under ``ACCEPTED``. + """ + + accepted: bool + reason: str + candidate_sha: str + champion_sha: str + seeds: tuple[int, ...] + regression_passed: bool + champion_metrics: Metrics + challenger_metrics: Metrics + + def __post_init__(self) -> None: + if self.accepted and not self.regression_passed: + raise EvaluationError( + "accepted report cannot carry a failed regression suite" + ) + if self.accepted and self.reason != ACCEPTED: + raise EvaluationError(f"accepted report cannot read {self.reason!r}") + if not self.accepted and self.reason == ACCEPTED: + raise EvaluationError(f"rejected report cannot read {ACCEPTED!r}") + + +#: One reading's reason literal, champion mean, challenger mean and drop +#: threshold — everything a single comparison needs, in one place. +_Comparison = tuple[str, float, float, float] + + +@dataclass(frozen=True, slots=True) +class _Means: + """The un-rounded seed means, before the report rounds three of them.""" + + tool_errors: float + call_count: float + tokens: float + latency_s: float + + +def _means(readings: Sequence[Metrics]) -> _Means: + """Arithmetic mean of *readings*, field by field, without rounding.""" + count = len(readings) + return _Means( + tool_errors=sum(reading.tool_errors for reading in readings) / count, + call_count=sum(reading.call_count for reading in readings) / count, + tokens=sum(reading.tokens for reading in readings) / count, + latency_s=sum(reading.latency_s for reading in readings) / count, + ) + + +def _reported(means: _Means) -> Metrics: + """The reportable form: counts rounded, ``latency_s`` kept in seconds.""" + return Metrics( + tool_errors=round(means.tool_errors), + call_count=round(means.call_count), + tokens=round(means.tokens), + latency_s=means.latency_s, + ) + + +def _comparisons(champion: _Means, challenger: _Means) -> tuple[_Comparison, ...]: + """The four comparisons, in the order the worse-field scan must use.""" + return ( + ( + WORSE_TOOL_ERRORS, + champion.tool_errors, + challenger.tool_errors, + DROP_TOOL_ERRORS, + ), + (WORSE_CALL_COUNT, champion.call_count, challenger.call_count, DROP_CALL_COUNT), + (WORSE_TOKENS, champion.tokens, challenger.tokens, DROP_TOKENS), + (WORSE_LATENCY, champion.latency_s, challenger.latency_s, DROP_LATENCY_S), + ) + + +def _first_worse(comparisons: Sequence[_Comparison]) -> str | None: + """The reason named by the first regressing reading, or ``None``.""" + for reason, champion, challenger, drop in comparisons: + if challenger > champion + drop: + return reason + return None + + +def _has_gain(comparisons: Sequence[_Comparison]) -> bool: + """Whether any single reading improved beyond its drop threshold.""" + return any( + challenger < champion - drop for _, champion, challenger, drop in comparisons + ) + + +def evaluate( + challenger: Challenger, + challenger_tree: Path, + champion_sha: str, + held_out_cases: Sequence[EvalCase], + regression_cases: Sequence[EvalCase], + *, + runner: ContractRunner, + replay: ReplayFn, + seeds: Sequence[int] = DEFAULT_SEEDS, +) -> EvaluationReport: + """Decide whether *challenger* may replace the champion, and say why. + + The gate short-circuits in four steps: + + 1. A graduated regression case fails on ``challenger_tree`` → + rejected as ``REGRESSION_FAILED``, ``replay`` is never called and + both sides' metrics are zero. An empty ``regression_cases`` is + legal — nothing has graduated yet — and passes. + 2. Otherwise the held-out cases are replayed once per seed on each + side and the seed means compared reading by reading. The first + reading worse than the champion by more than its ``DROP_*`` + threshold names the reason, in the order ``tool_errors`` → + ``call_count`` → ``tokens`` → ``latency_s``. A gain elsewhere + never offsets it: there is no total to trade in. + 3. Nothing worse but nothing better either → ``NO_PRACTICAL_GAIN``. + 4. Otherwise accepted. + + Both comparisons run on the un-rounded means; the report rounds the + three counts only on the way in, and keeps ``latency_s`` as the mean + in seconds. + + Args: + challenger: The checkout under evaluation. Only its ``sha`` is + read, and only to fill ``candidate_sha``. + challenger_tree: Tree the challenger is checked out in. Handed to + ``runner`` and ``replay``; never opened or stat'd here. + champion_sha: Full sha of the champion. Handed to ``replay``, + which owns resolving it to a tree. + held_out_cases: Cases replayed on both sides. Must not be empty. + regression_cases: Graduated cases run against the challenger + tree only. May be empty. + runner: Seam that runs ``regression_cases``. Keyword-only with no + default: the production runner needs a host. + replay: Seam that replays ``held_out_cases`` for one side under + one seed. Keyword-only with no default, for the same reason. + seeds: Seeds to replay under, once each per side. Defaults to + :data:`DEFAULT_SEEDS`; must not be empty. + + Returns: + One :class:`EvaluationReport` carrying the verdict, its reason, + the seeds used and both sides' mean metrics. No pointer is read + or written: promotion is a later leaf's job. + + Raises: + EvaluationError: ``seeds`` or ``held_out_cases`` is empty. Raised + before ``runner`` or ``replay`` is called, so no report and + no side effect comes of it. + """ + if not seeds: + raise EvaluationError("evaluate needs at least one seed") + if not held_out_cases: + raise EvaluationError("evaluate needs at least one held-out case") + + if regression_cases and not runner.run(challenger_tree, regression_cases).passed: + return EvaluationReport( + accepted=False, + reason=REGRESSION_FAILED, + candidate_sha=challenger.sha, + champion_sha=champion_sha, + seeds=tuple(seeds), + regression_passed=False, + champion_metrics=_ZERO_METRICS, + challenger_metrics=_ZERO_METRICS, + ) + + champion_means = _means( + tuple(replay(champion_sha, held_out_cases, seed) for seed in seeds) + ) + challenger_means = _means( + tuple(replay(challenger_tree, held_out_cases, seed) for seed in seeds) + ) + + comparisons = _comparisons(champion_means, challenger_means) + reason = _first_worse(comparisons) + if reason is None: + reason = ACCEPTED if _has_gain(comparisons) else NO_PRACTICAL_GAIN + + return EvaluationReport( + accepted=reason == ACCEPTED, + reason=reason, + candidate_sha=challenger.sha, + champion_sha=champion_sha, + seeds=tuple(seeds), + regression_passed=True, + champion_metrics=_reported(champion_means), + challenger_metrics=_reported(challenger_means), + ) diff --git a/src/molmcp/gate.py b/src/molmcp/gate.py new file mode 100644 index 0000000..1ae31c2 --- /dev/null +++ b/src/molmcp/gate.py @@ -0,0 +1,634 @@ +"""``molmcp gate`` — the wiring contract, and nothing else. + +This repository has one required GitHub check, ``official/gate``, and the +same sentence runs it in three places: the pull-request job's ``run:``, the +schedule job's ``run:``, and the pre-push hook's ``entry:``. All three are +serialized copies of :data:`GATE_RUN`; the authority is the constant here. +:func:`run_gate` reads the two files those copies live in and reports every +place they have drifted apart. + +What it deliberately does not do is run anything. Lint and tests belong to +``ci.yml``'s OS/Python matrix, and a gate that shelled out to them would be +a second, slower copy of that matrix which could disagree with the first. +So this module spawns no process, and it reads no environment either: a +gate configured from outside decides one thing on a laptop and another on a +runner, which is exactly the disagreement it exists to catch. + +There is one profile. An earlier draft had a ``--full`` that called an +evaluation, but an evaluation needs two subagents and a GitHub runner has +none, so it could never have run where it was wired. Every parameter that +would have selected a profile is gone: :func:`run_gate` takes ``root`` and +that is the whole signature. + +A root missing half the contract is a verdict, not a traceback — a report +saying which file is absent is actionable, and an ``OSError`` out of a +required check only says the check itself broke. Each message names the +offending token, so a reader can act on it without opening both files. + +The YAML reading here is a scanner over a known shape, not a parser: these +two files are written by this repository, and every value it needs is a +scalar or a flow sequence on one line. Adding a YAML dependency to compare +four strings would be a runtime dependency for the gate that guards the +runtime. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path +from typing import NamedTuple + +__all__ = [ + "CHECK_NAME", + "GATE_RUN", + "PR_JOB_ID", + "SCHEDULE_JOB_ID", + "GateReport", + "run_gate", +] + +#: The GitHub required check name — the pull-request job's ``name:``, not +#: its id. GitHub matches a required check on the name it displays. +CHECK_NAME = "official/gate" + +#: Id of the pull-request job. Never ``gate``: +#: ``.github/workflows/release.yml`` already owns that id for the release +#: gate, and two jobs answering to one id is a rename waiting to happen. +PR_JOB_ID = "official-gate" + +#: Id of the scheduled job. It runs the same literal on a timer, so that a +#: wiring broken between pull requests is still found within the week. +SCHEDULE_JOB_ID = "official-gate-schedule" + +#: The one call literal. The workflow's two ``run:`` steps and the +#: pre-commit hook's ``entry:`` are copies of this string, compared +#: character for character. +GATE_RUN = "uv run molmcp gate" + +#: The workflow half of the contract, relative to the repository root. +WORKFLOW_PATH = ".github/workflows/official-gate.yml" + +#: The pre-commit half of the contract, relative to the repository root. +PRE_COMMIT_PATH = ".pre-commit-config.yaml" + +#: Id of the pre-commit hook carrying :data:`GATE_RUN`. The same word as +#: :data:`PR_JOB_ID` on purpose: one check, one name everywhere. +HOOK_ID = PR_JOB_ID + +#: The hook the commit stage keeps. The gate is a pre-push hook; a commit +#: stage that grew a second slow hook is a wiring change, not a preference. +COMMIT_HOOK_ID = "ci-lint" + +#: pre-commit's name for the push stage, and the gate hook's only stage. +PRE_PUSH_STAGE = "pre-push" + +#: pre-commit's name for the commit stage. +COMMIT_STAGE = "pre-commit" + +#: The pull-request job's guard: it is the check, so it runs for everything +#: except the timer. +PR_IF = "github.event_name != 'schedule'" + +#: The scheduled job's guard, the complement of :data:`PR_IF`, so that one +#: event never runs both jobs. +SCHEDULE_IF = "github.event_name == 'schedule'" + +#: A GitHub expression. Legal in ``if:`` and ``concurrency:``, forbidden in +#: a ``run:``: what an expression expands to is not what parity compared. +EXPRESSION = "${{" + +#: How the gate step is recognised before its literal is compared — the two +#: words no other step in either job carries. Not a second call literal: it +#: selects which ``run:`` to read, and the reading is against +#: :data:`GATE_RUN`. +_GATE_CALL = "molmcp gate" + +#: YAML's block scalar indicators. A gate call written as a block is not a +#: single-line literal, whatever its body says. +_BLOCK_INDICATORS = frozenset({"|", "|-", "|+", ">", ">-", ">+"}) + +#: Quote characters a scalar may be wrapped in. +_QUOTES = "\"'" + + +@dataclass(frozen=True, slots=True) +class GateReport: + """The verdict of one :func:`run_gate` call. + + Attributes: + ok: Whether every checked copy of the contract still agrees. + failed: One message per disagreement, each naming the file and the + offending token. Empty exactly when ``ok`` is ``True``. + """ + + ok: bool + failed: tuple[str, ...] + + +class _Line(NamedTuple): + """One significant line of a scanned file. + + Attributes: + number: 1-based line number, used to name a failure's location. + indent: Leading spaces, which is what nesting means in these files. + text: The line with surrounding whitespace removed. + """ + + number: int + indent: int + text: str + + +class _Run(NamedTuple): + """One ``run:`` step of a job. + + Attributes: + number: Line number of the ``run:`` key. + text: The single-line scalar, or the joined body of a block scalar. + block: Whether the value was written as a block rather than inline. + """ + + number: int + text: str + block: bool + + +def _unquote(value: str) -> str: + """Strip one matching pair of surrounding quotes from *value*. + + Args: + value: A scalar as written, already stripped of whitespace. + + Returns: + The scalar without its wrapping quotes. Quotes inside an unquoted + value — ``github.event_name != 'schedule'`` — are left alone. + """ + if len(value) >= 2 and value[0] == value[-1] and value[0] in _QUOTES: + return value[1:-1] + return value + + +def _significant_lines(path: Path) -> tuple[_Line, ...]: + """Read *path*, dropping blank lines and whole-line comments. + + Args: + path: File to read, known to exist. + + Returns: + Every remaining line with its number and indent. + """ + lines: list[_Line] = [] + for number, raw in enumerate(path.read_text(encoding="utf-8").splitlines(), 1): + text = raw.strip() + if not text or text.startswith("#"): + continue + lines.append(_Line(number, len(raw) - len(raw.lstrip(" ")), text)) + return tuple(lines) + + +def _key_value(line: _Line) -> tuple[str, str] | None: + """Split *line* into a mapping key and its scalar. + + A leading ``- `` is dropped first, so the first key of a list item reads + like any other key. A key containing a space is not a key: that is a + line of shell inside a block scalar. + + Args: + line: A significant line. + + Returns: + The key and its unquoted scalar, or ``None`` when the line is not a + mapping entry. The scalar is empty when the value is nested below. + """ + text = line.text[2:].lstrip() if line.text.startswith("- ") else line.text + key, separator, value = text.partition(":") + if not separator or not key or " " in key: + return None + return key, _unquote(value.strip()) + + +def _children(lines: tuple[_Line, ...], index: int) -> tuple[_Line, ...]: + """The lines nested under ``lines[index]``. + + Args: + lines: The block being scanned. + index: Position of the parent line. + + Returns: + Every following line indented deeper than the parent, up to the + first that is not. + """ + parent = lines[index].indent + end = index + 1 + while end < len(lines) and lines[end].indent > parent: + end += 1 + return lines[index + 1 : end] + + +def _field(block: tuple[_Line, ...], key: str) -> tuple[_Line, str] | None: + """Look *key* up among the direct children of *block*. + + Args: + block: The nested lines of one mapping. + key: Mapping key to find. + + Returns: + The line carrying *key* and its scalar, or ``None``. Only the + shallowest lines of *block* are direct children; a deeper ``name:`` + belongs to a step, not to the job. + """ + if not block: + return None + depth = min(line.indent for line in block) + for line in block: + if line.indent != depth: + continue + pair = _key_value(line) + if pair is not None and pair[0] == key: + return line, pair[1] + return None + + +def _sequence(block: tuple[_Line, ...], key: str) -> tuple[str, ...] | None: + """Read *key* of *block* as a list, flow or nested. + + Args: + block: The nested lines of one mapping. + key: Mapping key to find. + + Returns: + The items in order, or ``None`` when *key* is absent. + """ + found = _field(block, key) + if found is None: + return None + line, value = found + if value.startswith("[") and value.endswith("]"): + inner = value[1:-1].strip() + return ( + tuple(_unquote(item.strip()) for item in inner.split(",")) if inner else () + ) + if value: + return (value,) + return tuple( + _unquote(child.text[2:].strip()) + for child in _children(block, block.index(line)) + if child.text.startswith("- ") + ) + + +def _jobs(lines: tuple[_Line, ...]) -> dict[str, tuple[_Line, ...]]: + """Every job of a workflow, by id. + + Args: + lines: The significant lines of a workflow file. + + Returns: + Each job id mapped to the lines nested under it, in file order. + Empty when the file has no top-level ``jobs:``. + """ + block: tuple[_Line, ...] = () + for index, line in enumerate(lines): + if line.indent == 0 and _key_value(line) == ("jobs", ""): + block = _children(lines, index) + break + if not block: + return {} + depth = min(line.indent for line in block) + jobs: dict[str, tuple[_Line, ...]] = {} + for index, line in enumerate(block): + pair = _key_value(line) if line.indent == depth else None + if pair is not None: + jobs[pair[0]] = _children(block, index) + return jobs + + +def _runs(block: tuple[_Line, ...]) -> tuple[_Run, ...]: + """Every ``run:`` anywhere inside a job. + + Args: + block: The lines nested under one job. + + Returns: + One :class:`_Run` per ``run:`` key, in file order. + """ + runs: list[_Run] = [] + for index, line in enumerate(block): + pair = _key_value(line) + if pair is None or pair[0] != "run": + continue + value = pair[1] + if value and value not in _BLOCK_INDICATORS: + runs.append(_Run(line.number, value, False)) + continue + body = " ".join(child.text for child in _children(block, index)) + runs.append(_Run(line.number, body, True)) + return tuple(runs) + + +def _hooks(lines: tuple[_Line, ...]) -> dict[str, tuple[_Line, ...]]: + """Every pre-commit hook, by id. + + Args: + lines: The significant lines of a pre-commit config. + + Returns: + Each hook id mapped to the lines nested under its list item. + """ + hooks: dict[str, tuple[_Line, ...]] = {} + for index, line in enumerate(lines): + if not line.text.startswith("- "): + continue + pair = _key_value(line) + if pair is not None and pair[0] == "id": + hooks[pair[1]] = _children(lines, index) + return hooks + + +def _check_gate_call(job_id: str, block: tuple[_Line, ...]) -> tuple[str, ...]: + """Check that the job calls the gate by the one literal. + + Args: + job_id: Id of the job, so a failure names which one. + block: The lines nested under that job. + + Returns: + A message per offending step: none found, written as a block, or + written as some other string. + """ + calls = [run for run in _runs(block) if _GATE_CALL in run.text] + if not calls: + return ( + f"{WORKFLOW_PATH}: job {job_id!r} has no step whose " + f"run: is {GATE_RUN!r} (the Install step is a prior step, " + f"not the compared token).", + ) + failed: list[str] = [] + for run in calls: + if run.block: + failed.append( + f"{WORKFLOW_PATH}: job {job_id!r} line {run.number}: " + f"run: is a block scalar; the gate call is the single-line " + f"run: {GATE_RUN!r}." + ) + elif run.text != GATE_RUN: + failed.append( + f"{WORKFLOW_PATH}: job {job_id!r} line {run.number}: " + f"run: {run.text!r} is not {GATE_RUN!r}." + ) + return tuple(failed) + + +def _check_literal_runs(job_id: str, block: tuple[_Line, ...]) -> tuple[str, ...]: + """Check that no ``run:`` of the job hides behind an expression. + + Args: + job_id: Id of the job, so a failure names which one. + block: The lines nested under that job. + + Returns: + A message per ``run:`` containing a GitHub expression. ``if:`` and + ``concurrency:`` may hold one; a ``run:`` may not, because what an + expression expands to on a runner is not what parity compared. + """ + return tuple( + f"{WORKFLOW_PATH}: job {job_id!r} line {run.number}: " + f"run: {run.text!r} contains {EXPRESSION!r}; a run: that expands " + f"is not a literal." + for run in _runs(block) + if EXPRESSION in run.text + ) + + +def _check_no_profile_env(job_id: str, block: tuple[_Line, ...]) -> tuple[str, ...]: + """Check that the job selects nothing from the environment. + + Args: + job_id: Id of the job, so a failure names which one. + block: The lines nested under that job. + + Returns: + A message per ``env:`` key found anywhere in the job. There is one + profile, so an ``env:`` here can only be selecting a second. + """ + return tuple( + f"{WORKFLOW_PATH}: job {job_id!r} line {line.number}: env: — the " + f"gate has one profile and selects nothing from the environment." + for line in block + if (pair := _key_value(line)) is not None and pair[0] == "env" + ) + + +def _check_condition( + job_id: str, block: tuple[_Line, ...], expected: str +) -> tuple[str, ...]: + """Check the job's ``if:`` guard. + + Args: + job_id: Id of the job, so a failure names which one. + block: The lines nested under that job. + expected: The guard this job must carry. + + Returns: + One message when the guard is absent or different, else nothing. + """ + found = _field(block, "if") + if found is None: + return (f"{WORKFLOW_PATH}: job {job_id!r} has no if:; expected {expected!r}.",) + if found[1] != expected: + return ( + f"{WORKFLOW_PATH}: job {job_id!r} line {found[0].number}: " + f"if: {found[1]!r} is not {expected!r}.", + ) + return () + + +def _check_pr_name(block: tuple[_Line, ...]) -> tuple[str, ...]: + """Check that the pull-request job is named after the required check. + + Args: + block: The lines nested under the pull-request job. + + Returns: + One message when the ``name:`` GitHub matches on is absent or is + not :data:`CHECK_NAME`, else nothing. + """ + found = _field(block, "name") + if found is None: + return ( + f"{WORKFLOW_PATH}: job {PR_JOB_ID!r} has no name:; the required " + f"check is matched on name: {CHECK_NAME!r}.", + ) + if found[1] != CHECK_NAME: + return ( + f"{WORKFLOW_PATH}: job {PR_JOB_ID!r} line {found[0].number}: " + f"name: {found[1]!r} is not the required check name " + f"{CHECK_NAME!r}.", + ) + return () + + +def _check_schedule_name(block: tuple[_Line, ...]) -> tuple[str, ...]: + """Check that the scheduled job does not claim the required check name. + + Args: + block: The lines nested under the scheduled job. + + Returns: + One message when the job is named :data:`CHECK_NAME`, else nothing. + Two jobs under one name would let a timer report the check that a + pull request is supposed to report. + """ + found = _field(block, "name") + if found is not None and found[1] == CHECK_NAME: + return ( + f"{WORKFLOW_PATH}: job {SCHEDULE_JOB_ID!r} line " + f"{found[0].number}: name: {CHECK_NAME!r} is the required check " + f"name; the scheduled job needs its own.", + ) + return () + + +def _check_workflow(root: Path) -> tuple[str, ...]: + """Check the workflow half of the contract under *root*. + + Args: + root: Repository root the relative paths are read from. + + Returns: + Every disagreement found, empty when the workflow is wired. + """ + path = root / WORKFLOW_PATH + if not path.is_file(): + return ( + f"{WORKFLOW_PATH} is missing: nothing runs {GATE_RUN!r} on a " + f"pull request, so the {CHECK_NAME!r} check reports nothing.", + ) + jobs = _jobs(_significant_lines(path)) + expected = (PR_JOB_ID, SCHEDULE_JOB_ID) + found = ", ".join(jobs) or "none" + failed = [ + f"{WORKFLOW_PATH}: no job with id {job_id!r} (jobs found: {found})." + for job_id in expected + if job_id not in jobs + ] + failed.extend( + f"{WORKFLOW_PATH}: unexpected job id {job_id!r}; this workflow holds " + f"{PR_JOB_ID!r} and {SCHEDULE_JOB_ID!r} and nothing else." + for job_id in jobs + if job_id not in expected + ) + for job_id, condition in ((PR_JOB_ID, PR_IF), (SCHEDULE_JOB_ID, SCHEDULE_IF)): + block = jobs.get(job_id) + if block is None: + continue + failed.extend(_check_condition(job_id, block, condition)) + failed.extend(_check_gate_call(job_id, block)) + failed.extend(_check_literal_runs(job_id, block)) + failed.extend(_check_no_profile_env(job_id, block)) + if PR_JOB_ID in jobs: + failed.extend(_check_pr_name(jobs[PR_JOB_ID])) + if SCHEDULE_JOB_ID in jobs: + failed.extend(_check_schedule_name(jobs[SCHEDULE_JOB_ID])) + return tuple(failed) + + +def _check_gate_hook(hook: tuple[_Line, ...]) -> tuple[str, ...]: + """Check the gate hook's ``entry:`` and stage. + + Args: + hook: The lines nested under the ``official-gate`` hook. + + Returns: + A message per disagreement. The ``entry:`` is the bare literal — + not ``entry: uv`` plus ``args:``, and not a + ``bash -c 'uv sync && …'`` wrapper, either of which is a different + string from the one the workflow runs. + """ + failed: list[str] = [] + entry = _field(hook, "entry") + if entry is None: + failed.append( + f"{PRE_COMMIT_PATH}: hook {HOOK_ID!r} has no entry:; expected " + f"entry: {GATE_RUN!r}." + ) + elif entry[1] != GATE_RUN: + failed.append( + f"{PRE_COMMIT_PATH}: hook {HOOK_ID!r} line {entry[0].number}: " + f"entry: {entry[1]!r} is not {GATE_RUN!r}." + ) + stages = _sequence(hook, "stages") + if stages != (PRE_PUSH_STAGE,): + failed.append( + f"{PRE_COMMIT_PATH}: hook {HOOK_ID!r} stages: " + f"{list(stages or ())} is not [{PRE_PUSH_STAGE}]; the gate runs " + f"before a push and the commit stage stays fast." + ) + return tuple(failed) + + +def _check_pre_commit(root: Path) -> tuple[str, ...]: + """Check the pre-commit half of the contract under *root*. + + Args: + root: Repository root the relative paths are read from. + + Returns: + Every disagreement found, empty when the hook is wired. + """ + path = root / PRE_COMMIT_PATH + if not path.is_file(): + return ( + f"{PRE_COMMIT_PATH} is missing: nothing runs {GATE_RUN!r} before " + f"a push, so the wiring is only checked once it is on GitHub.", + ) + hooks = _hooks(_significant_lines(path)) + failed: list[str] = [] + gate_hook = hooks.get(HOOK_ID) + if gate_hook is None: + failed.append( + f"{PRE_COMMIT_PATH}: no hook with id {HOOK_ID!r} carrying " + f"entry: {GATE_RUN!r}." + ) + else: + failed.extend(_check_gate_hook(gate_hook)) + commit_hook = hooks.get(COMMIT_HOOK_ID) + if commit_hook is None: + failed.append( + f"{PRE_COMMIT_PATH}: no hook with id {COMMIT_HOOK_ID!r}; the " + f"{COMMIT_STAGE!r} stage holds it and nothing else." + ) + else: + stages = _sequence(commit_hook, "stages") + if stages is None or COMMIT_STAGE not in stages: + failed.append( + f"{PRE_COMMIT_PATH}: hook {COMMIT_HOOK_ID!r} no longer lists " + f"the {COMMIT_STAGE!r} stage; the {COMMIT_STAGE!r} stage " + f"holds it and nothing else." + ) + return tuple(failed) + + +def run_gate(*, root: Path) -> GateReport: + """Decide whether the wiring contract under *root* still holds. + + Reads two files and compares four strings: the pull-request job's gate + ``run:``, the scheduled job's gate ``run:``, the pre-push hook's + ``entry:``, and the pull-request job's ``name:``. Nothing is executed, + no process is spawned, and no environment is read — lint and tests are + ``ci.yml``'s matrix, and a gate that read the environment would decide + differently on a laptop than on a runner. + + Args: + root: Repository root to read the contract from. Required and + keyword-only: the CLI passes ``Path.cwd()`` and tests pass a + fixture tree, so the gate never guesses which repository it is + judging. + + Returns: + A :class:`GateReport` whose ``failed`` names every disagreement, + file and token. A missing file is one of those messages, not an + exception: a required check that raises only says that it broke. + """ + failed = (*_check_workflow(root), *_check_pre_commit(root)) + return GateReport(ok=not failed, failed=failed) diff --git a/src/molmcp/harness.py b/src/molmcp/harness.py new file mode 100644 index 0000000..946cf0b --- /dev/null +++ b/src/molmcp/harness.py @@ -0,0 +1,659 @@ +"""The activated harness checkouts: bind the pointers, fold the catalogs, adapt. + +Serving from a harness is a read of the activation pointers and of each +checkout's ``harness.toml`` — never a fetch, never a write. Every source the +operator named owns its own pointer file beside one shared store, and the +components those catalogs declare are folded into one served set, first source +in the settings list winning a contested id. This module holds the arms that do +that reading; :mod:`molmcp.server` composes them, owns the decision of when to +run each one, and owns resolving the :class:`~molmcp.config.AppConfig` they are +handed. + +One name here is shared with :mod:`molmcp.harness_sync`, which does the +writing: :func:`assert_servable`, which entries an install may reach at all. It +lives on this side because a rule with two spellings is a rule two commands can +disagree about — a settings entry ``molmcp serve`` refuses cannot be one +``molmcp harness sync`` accepts. + +The rest of what those commands share is spelled in :mod:`molmcp.harness_paths` +and imported from there: :data:`~molmcp.harness_paths.SUPPORTED_CAPABILITIES`, +:func:`~molmcp.harness_paths.local_checkout_path` (which directory a local entry +names), :func:`~molmcp.harness_paths.store_path` and +:func:`~molmcp.harness_paths.pointer_path` (where a commit and its activation +land). They sit below this module rather than in it because a third command +needs them — ``molmcp init``, which mounts no plane and must not pay the import +cost this module's next paragraph describes just to read a pointer. + +This module sits on the **heavy** side of the child-safe import boundary, by +choice rather than by accident: it carries +``from .provider_worker.worker import WorkerProvider``, so importing it drags +the whole FastMCP-bearing worker stack into the importing process. +:mod:`molmcp.server` pays that cost already. Anything else reaching in here — +a CLI verb wanting the activated checkout, say — inherits it, and should know +that before reaching (``notes.md:worker-child-isolation``, which names +:mod:`molmcp.provider_sdk` and :mod:`molmcp.provider` as the boundary this +module is deliberately outside of). + +The cache root arrives through :func:`molmcp.runtime.resolved_cache_dir`, the +same shield :mod:`molmcp.server` uses, so nothing here imports +:mod:`molmcp.discovery`. +""" + +from __future__ import annotations + +import logging +from collections.abc import Sequence +from dataclasses import dataclass +from pathlib import Path + +from .components import ( + Activation, + CatalogError, + ComponentKind, + ComponentSpec, + GitHubTransport, + ImmutableGitStore, + load_harness_catalog, +) +from .config import AppConfig, ConfigurationError +from .harness_paths import ( + SUPPORTED_CAPABILITIES, + local_checkout_path, + pointer_path, + store_path, +) +from .provider import Provider +from .provider_worker.worker import WorkerProvider +from .runtime import resolved_cache_dir +from .settings import HarnessSource + +logger = logging.getLogger(__name__) + +#: What a local origin must have at the root it names. Probed with ``exists`` +#: rather than ``is_dir``: ``.git`` is a directory in an ordinary clone and a +#: file in a linked worktree, and both are checkouts. +_GIT_DIR_NAME = ".git" + +#: The one shared pointer file this install bound before sources were +#: activated by name. It is *named* once when it is the only pointer on disk +#: and never read. Nothing writes it: ``molmcp harness sync`` is now the one +#: caller of :meth:`~molmcp.components.Activation.stage` and +#: :meth:`~molmcp.components.Activation.promote` in ``src/``, and it writes +#: only through :func:`pointer_path`, which always interpolates a non-empty +#: source name. No source name can therefore produce this file, so the probe +#: is unambiguous — and the population it can still mislead is whoever ran a +#: pre-``sync`` build by hand, which is why one notice is the whole budget. +_LEGACY_POINTER_NAME = "harness.pointer" + + +@dataclass(frozen=True, slots=True) +class Checkout: + """The harness commit this process serves from, already on disk. + + Attributes: + sha: Activated commit SHA, as the pointer file records it. + tree: Root of that commit's tree — where ``harness.toml`` sits. + enable: Copy of :attr:`HarnessSource.enable` for this source. + ``None`` means every bundle; ``()`` means contribute no + members. Required so a forgotten argument cannot silently + serve nothing. + source: Name of the harness source this commit was activated for. + It rides here, beside ``tree``, so that ``source -> tree`` has + exactly one owner: :class:`ComponentFold` carries these objects + rather than a second mapping of the same fact. That ownership + survives :attr:`ComponentFold.component_roots`, and the + distinguishing fact is what that field maps to: it is + ``source -> str``, not ``source -> tree``. It records the + ``component_root`` string a catalog authored — a fact no + ``Checkout`` holds, because :func:`activated_checkouts` reads + no catalog — and :meth:`ComponentFold.root_for` joins it onto + the tree found *here*. The tree is therefore still owned once, + and a base built against some other source's tree is not a + state that type can hold. + """ + + sha: str + tree: Path + source: str + enable: tuple[str, ...] | None + + +@dataclass(frozen=True, slots=True) +class SourcedComponent: + """One catalog component paired with the source it was declared in. + + The cross-source key is this *pair*, and ``spec.id`` is carried + **unchanged**: a component out of a source named ``official`` still has + ``spec.id == "provider.demo"``. The pair does not try to namespace the id, + because a namespaced id is not constructible — + :class:`~molmcp.components.ComponentSpec` pins ``id == f"{kind}.{name}"`` + and its member pattern admits only the five known kinds, so + ``official.provider.demo`` is refused at construction. + + The shape is :class:`~molmcp.collection.models.SearchHit`'s, which keeps + ``source`` as a field *beside* the ref and never folds one into the other + — this repo's existing answer to "the same id from two origins". + + Attributes: + source: Name of the harness source the spec came from, as the + ``harness`` settings list spells it. + spec: The catalog row, exactly as its catalog declared it. + """ + + source: str + spec: ComponentSpec + + +@dataclass(frozen=True, slots=True) +class ComponentFold: + """The result of folding one component kind over several checkouts. + + There is deliberately no ``displaced`` field. A component that lost a + contested id is *reported* — one warning from this module's logger — not + stored: nothing in production would read such a field, and this repo's + other first-wins folds (``discovery/overlay/catalog.py``, + ``discovery/overlay/conventions.py``) drop losers without recording them. + + ``__post_init__`` refuses any fold whose two collections disagree: source + names are unique on **both** sides and exactly equal across them. Each + clause earns its keep. Set equality alone admits one source named twice + in ``component_roots``, and :meth:`root_for`'s scan would then answer + with whichever entry it met first while a second entry said something + else. Set equality *and* roots-side uniqueness together still admit two + checkouts sharing one source name with different trees, where the scan + answers with the first tree and the other source's components resolve + nowhere. :func:`activated_checkouts` already refuses a duplicate source + name, but this type is directly constructible and cannot rely on its own + caller. The guard owns the **correspondence** between the two + collections, which nothing else owns, and deliberately does *not* + re-validate the ``component_root`` string: that **value**'s one home is + :class:`~molmcp.components.HarnessCatalog`'s own ``__post_init__``, and a + second guard here would be a second owner of one rule. + + Attributes: + checkouts: The checkouts this fold was built from, in source order. + The fold carries the objects themselves rather than a parallel + ``source -> tree`` map, so a consumer that needs a kept spec's + tree has one place to find it and cannot hold two arguments out + of sync. + component_roots: Each source's ``component_root`` exactly as its + catalog authored it — the raw string, in source order, ``""`` + for a catalog declaring no key. This is not the parallel map the + entry above forbids, and both halves of why are worth stating. + The **string** is the fold's own datum: no ``Checkout`` holds + it, because :func:`activated_checkouts` reads no catalog, and + giving ``Checkout`` the field would force it to. The **join** is + what would have been the duplicate — ``tree`` already lives on + the checkouts, so storing ``tree / component_root`` would be a + second copy of a fact those objects already hold, and the + invariant above would then exist only to police the agreement + between two copies of one fact. With the string stored and the + join performed inside :meth:`root_for` against *that source's + own* :attr:`Checkout.tree`, "the base belongs to the right tree" + is a theorem rather than an assertion: there is no other tree + :meth:`root_for` can reach. + kept: The surviving components — source order outside, catalog order + within a source. + + Raises: + CatalogError: ``checkouts`` and ``component_roots`` disagree — a + source name repeats on either side, or the two sets of names are + not equal. + """ + + checkouts: tuple[Checkout, ...] + component_roots: tuple[tuple[str, str], ...] + kept: tuple[SourcedComponent, ...] + + def __post_init__(self) -> None: + """Refuse a fold that cannot answer exactly one base per source. + + Raises: + CatalogError: A source name repeats among ``checkouts``, or + repeats among ``component_roots``, or the two collections + do not name the same set of sources. + """ + checked = tuple(checkout.source for checkout in self.checkouts) + rooted = tuple(source for source, _ in self.component_roots) + if len(set(checked)) != len(checked): + raise CatalogError( + f"a component fold cannot hold two checkouts of one harness " + f"source: {sorted(checked)!r}. `root_for` would answer with " + f"the first one's tree and the second's components would " + f"resolve nowhere." + ) + if len(set(rooted)) != len(rooted): + raise CatalogError( + f"a component fold cannot hold two component roots for one " + f"harness source: {sorted(rooted)!r}. `root_for` would answer " + f"with the first one and the second would be silently unused." + ) + if set(checked) != set(rooted): + raise CatalogError( + f"a component fold must hold exactly one component root per " + f"checkout: its checkouts name {sorted(checked)!r} and its " + f"component roots name {sorted(rooted)!r}." + ) + + def root_for(self, source: str) -> Path: + """Return the directory *source*'s component paths resolve under. + + This is the **one** place a checkout tree and a catalog's + ``component_root`` are joined, and it joins them against that + source's own :attr:`Checkout.tree` — so a base belonging to another + source's tree is not reachable rather than merely untested. A + catalog declaring no ``component_root`` answers the tree object + itself, not another spelling of it: no ``.`` component, no trailing + separator, so an install that has no key today resolves + byte-identical paths. + + A linear scan, symmetric with :meth:`specs_from` — but deliberately + **not** symmetric with its tolerance of an unknown source. There is + no empty ``Path`` a caller could stand in with, and a wrong base is + the half-applied failure this whole design exists to prevent. + + Args: + source: Harness source name, as the ``harness`` settings list + spells it. + + Returns: + ``tree / component_root`` for a rooted source, and exactly + ``tree`` for a rootless one. + + Raises: + CatalogError: This fold was not built from that source. The + message contains ``unknown-source`` and names it with + ``repr`` — the register :meth:`HarnessCatalog.get + ` and ``get_bundle`` + already use. + """ + for name, component_root in self.component_roots: + if name != source: + continue + for checkout in self.checkouts: + if checkout.source != source: + continue + if not component_root: + return checkout.tree + return checkout.tree / component_root + raise CatalogError(f"unknown-source: {source!r}") + + @property + def names(self) -> frozenset[str]: + """Component names of every kept component. + + For :attr:`~molmcp.components.ComponentKind.PROVIDER` these are the + plane ids clients see and the names entry-point planes are XORed + against. A contested id appears once, because only its winner is + kept — which is what stops two planes mounting under one namespace. + """ + return frozenset(sourced.spec.name for sourced in self.kept) + + def specs_from(self, source: str) -> tuple[ComponentSpec, ...]: + """Return the specs *source* kept, in that catalog's own order. + + Grouping is per source because every consumer of a spec also needs + the tree it came from: an import root, an overlay's seed path. A spec + that lost a contested id is not kept, so its own source does not + report it either. + + Args: + source: Harness source name to select. + + Returns: + That source's kept specs in catalog order, or the empty tuple + when the source kept nothing — including when it was never + folded at all. An unknown source is not an error; it is a source + with nothing in it. + """ + return tuple(sourced.spec for sourced in self.kept if sourced.source == source) + + +def assert_servable(source: HarnessSource) -> None: + """Refuse one harness source that names no origin this install can reach. + + A GitHub locator is complete without a path, including with an empty + ``ref`` — :meth:`~molmcp.components.GitHubTransport.resolve_commit` + treats ``None`` as the repository default. A local locator must name a + checkout already on disk: ``.git`` exists under + :func:`~molmcp.harness_paths.local_checkout_path`. ``enable`` is not + read; catalog filtering is a later slice. + + Expanding is not rewriting. The source is read and never modified: the + stored locator is the operator's, and every message here reports it + **as written**. + + This is the single owner of the rule. ``molmcp serve`` reaches it through + :func:`molmcp.server._harness_locator` and ``molmcp harness sync`` calls + it on the one entry it was given, so an entry one command refuses cannot + be one the other accepts. A local entry this lets through names a + checkout **once expanded**, which means calling + :func:`local_checkout_path` rather than opening ``source.path`` as + written. + + Args: + source: One entry of the ``harness`` settings list, as written. + + Raises: + ConfigurationError: The entry is local and does not name a git + checkout. The message names the entry, because under a list of + sources the entry's name is the address an operator goes to + fix it, and names the locator as the settings file spells it. + """ + if not source.is_local: + return + root = local_checkout_path(source) + if (root / _GIT_DIR_NAME).exists(): + return + raise ConfigurationError( + f"the harness source named {source.name!r} names a `path` that is " + f"not a git checkout: {source.locator}. A local origin is pinned to a " + f"commit exactly as a remote one is, so it must be the root of a " + f"repository already on disk — the directory holding its `.git`. " + f"Point that entry of the `harness` list at a checkout, or remove " + f"the entry to serve without it." + ) + + +def servable_sources( + sources: Sequence[HarnessSource], +) -> tuple[HarnessSource, ...]: + """Check every named source and hand the whole list back in file order. + + Every entry is checked and none is ever skipped. An entry that names no + reachable origin is refused rather than passed over in favour of its + neighbour, for the same reason no coordinate is defaulted: carrying on + from the next entry would serve code from a repository the operator did + not select. + + Args: + sources: Every named harness source, in the order the settings list + names them. + + Returns: + The same sources, in the same order — that order is the operator's + priority control over a component two sources both declare, and it is + carried through :func:`activated_checkouts` into the fold. + + Raises: + ConfigurationError: Any entry fails :func:`assert_servable`. + """ + for source in sources: + assert_servable(source) + return tuple(sources) + + +def activated_checkouts( + config: AppConfig, sources: Sequence[HarnessSource] +) -> tuple[Checkout, ...]: + """Bind one activation pointer per named source and return what they point at. + + Serving is a read of the pointers, never a write to one: each is bound, + its ``current`` is read, and that is the end of it. Staging, promoting and + fetching a commit belong to the commands that were asked to change what is + activated. + + One store and one transport are shared across every source, and only the + *pointers* multiply. :class:`ImmutableGitStore` keys a commit on its SHA + alone and records provenance per SHA, and :class:`GitHubTransport` takes + ``(owner, repo)`` per call, so a second root would buy no isolation and + would strand every already-published tree. A pointer is not shareable the + same way: the record it holds carries one ``active`` SHA, so two sources + folded into one file would overwrite each other's commit. + + A source with no pointer file, or with a pointer that activates nothing, + contributes no checkout and is not an error. Nothing activated serves + exactly like an unset locator, and it does so *per source*: the neighbour + still yields its own checkout. + + No catalog is read here. This function binds pointers and hands back trees; + what a tree declares is :func:`fold_components`' subject, and an arm that + does not run never pays for a catalog it would not use. + + Args: + config: **Already-resolved** application configuration. + :func:`~molmcp.server.create_stack` resolves it and passes it in so + that the store and every pointer land under the very same cache + root the collection indexes under. Resolution is that caller's job + and is deliberately not repeated here. + sources: Every named harness source, in the order the ``harness`` + settings list names them. That order is carried through to the + returned checkouts, and it is the operator's only priority + control: :func:`fold_components` resolves a contested component id + first-wins over this sequence. + + Returns: + One checkout per *activated* source, in source order. The empty tuple + when none of them is activated — the same answer as no source at all. + + Raises: + ConfigurationError: A source cannot be served. Three ways: its name + cannot name a pointer file (see :func:`pointer_path`); two entries + share a name, compared with ``casefold`` because both spellings + resolve to one file on darwin and on Windows; or its pointer names + a commit with no published tree. That last one is named rather + than silently re-fetched — serving a different commit than the one + that was activated is the one outcome nobody asked for — and it + names the *source*, because under N sources a SHA and a store root + identify no entry of the settings file to go and fix. + ActivationVersionError: A pointer file exists and is not a version-1 + activation record (bad JSON, unknown version, missing fields). + Raised by :meth:`Activation.bind`; a *missing* file is not an + error, it is the empty record that skips its source above. + """ + root = resolved_cache_dir(config) + # Every name is turned into a path before anything is bound, so a settings + # file this function refuses is refused whole rather than half-served. + pointers: list[tuple[HarnessSource, Path]] = [] + claimed: dict[str, str] = {} + for source in sources: + key = source.name.casefold() + first = claimed.get(key) + if first is not None: + raise ConfigurationError( + f"the `harness` list names two sources that own one " + f"activation pointer file: {first!r} and {source.name!r}. " + f"Names are compared case-insensitively because darwin and " + f"Windows resolve both spellings to the same file, so the " + f"second entry would silently serve whatever the first " + f"activated. Rename or remove one of those two entries in " + f"your settings file." + ) + claimed[key] = source.name + pointers.append((source, pointer_path(root, source.name))) + + legacy = root / _LEGACY_POINTER_NAME + if legacy.exists() and not any(pointer.exists() for _, pointer in pointers): + logger.warning( + "the activation pointer %s is left over from before this install " + "activated harness sources by name, and is never read: each " + "source now owns a `harness..pointer` file beside it, and " + "none of the named sources has one, so nothing is activated. " + "Delete that file, and activate the sources you want under their " + "own names.", + legacy, + ) + + store_root = store_path(root) + store = ImmutableGitStore(root=store_root, transport=GitHubTransport()) + checkouts: list[Checkout] = [] + for source, pointer in pointers: + activation = Activation.bind( + pointer, + store=store, + supported_capabilities=SUPPORTED_CAPABILITIES, + ) + current = activation.current + if current is None: + continue + if not store.has(current): + raise ConfigurationError( + f"the harness source named {source.name!r} is activated at " + f"commit {current}, which has no published tree under " + f"{store_root}. Publish and activate it again, or clear that " + f"source's activation pointer at {pointer}." + ) + checkouts.append( + Checkout( + sha=current, + tree=store.tree_path(current), + source=source.name, + enable=source.enable, + ) + ) + return tuple(checkouts) + + +def fold_components( + checkouts: Sequence[Checkout], kind: ComponentKind +) -> ComponentFold: + """Fold one component kind over every checkout, first source wins. + + Each checkout's ``harness.toml`` is the only inventory of its tree — the + tree is never globbed, because a file nobody declared is not a component — + and the catalogs are read here, in the order the ``harness`` settings list + names their sources. This is the one folder: each arm calls it for itself + with the kind it wants, so an arm that does not run never pays for a + catalog it would not use. When two sources declare the same ``spec.id``, **the + first one in that list keeps it** — the ``setdefault`` idiom this repo + already folds ordered streams with — and the later declaration is + dropped with one warning naming the winning source, the losing source + and the contested id. The order of the list is therefore the operator's + priority control, and the only one: there is no per-source override. + + Keying on ``spec.id`` rather than on the component name is what closes + the mount hazard for providers, where ``id == f"provider.{name}"`` makes + an id collision a plane-name collision: two sources shipping + ``provider.demo`` would otherwise build two planes named ``demo`` and + mount both under one namespace. + + Args: + checkouts: The activated checkouts, in source order. + kind: The one component kind to fold; every other kind in every + catalog is passed over. + + Returns: + The fold: the checkouts it was built from, each source's + ``component_root`` as its catalog authored it, and the components + that survived. No checkout at all is not an error — it is the empty + fold. The roots are recorded in the same loop iteration that reads + the catalog, because that iteration is the only place both the + source name and its catalog are in hand at once. + + Raises: + CatalogError: A catalog is malformed, or requires a capability this + runtime does not support. One bad catalog fails the serve rather + than being skipped in favour of its neighbours, for the same + reason an incomplete source is refused: carrying on would serve + code the operator did not select. Also when *checkouts* names one + source twice, which :class:`ComponentFold` refuses — + :func:`activated_checkouts` cannot produce that, but this + function is directly callable with a hand-built sequence. + """ + kept: dict[str, SourcedComponent] = {} + component_roots: list[tuple[str, str]] = [] + for checkout in checkouts: + catalog = load_harness_catalog( + checkout.tree, checkout.sha, SUPPORTED_CAPABILITIES + ) + component_roots.append((checkout.source, catalog.component_root)) + for spec in catalog.enabled_components(checkout.enable): + if spec.kind is not kind: + continue + sourced = SourcedComponent(source=checkout.source, spec=spec) + winner = kept.setdefault(spec.id, sourced) + if winner is sourced: + continue + logger.warning( + "the harness source named %r also declares %r, which the " + "source named %r declares first; the earlier entry of the " + "`harness` list wins, so %r's copy is served and this one " + "is ignored. Reorder that list, or drop the component from " + "one of the two catalogs.", + checkout.source, + spec.id, + winner.source, + winner.source, + ) + return ComponentFold( + checkouts=tuple(checkouts), + component_roots=tuple(component_roots), + kept=tuple(kept.values()), + ) + + +def checkout_planes(fold: ComponentFold) -> list[Provider]: + """Adapt a provider fold's kept components into mountable planes. + + Each one becomes a :class:`~molmcp.provider_worker.worker.WorkerProvider` + named by the component's ``name`` — the plane id clients see and the name + the entry-point comparison is made on. The component ``id`` + (``provider.demo``) is a catalog key, not a plane id; mounting under it + would namespace the plane's tools as ``provider.demo_open``. + + The fold is the *only* argument, deliberately. Every spec needs the base + its source resolves under to find its import root, and the fold answers + that itself through :meth:`ComponentFold.root_for` — so there is nothing + for a caller to keep in sync, and a fold built from some other checkout + list cannot be paired with a stale one here. Only kept components are + built, which is what stops two sources' ``provider.demo`` from mounting + twice under one namespace. + + Args: + fold: A :data:`~molmcp.components.ComponentKind.PROVIDER` fold. Any + other kind yields planes whose entrypoints were never meant to be + run in a worker; folding the right kind is the caller's business, + as it is the caller that named the kind. + + Returns: + One plane per kept provider component — source order outside, catalog + order within a source. Empty when nothing is activated. + """ + return [ + WorkerProvider( + # A provider component always carries an entrypoint — ComponentSpec + # refuses to be built without one — and it stays a string here: the + # checkout is imported in the child process, never in this one. + name=spec.name, + entrypoint=str(spec.entrypoint), + path=_import_root(fold.root_for(checkout.source), spec.path), + ) + for checkout in fold.checkouts + for spec in fold.specs_from(checkout.source) + ] + + +def _import_root(base: Path, path: str) -> Path: + """Resolve a component path to the directory its module is imported from. + + A component may point at either the module file (``providers/demo/plane.py``) + or the package directory that holds it (``providers/demo``). Both name the + same import root, so a directory is used as it stands and a file hands back + its parent. + + The overlay arm resolves its own import root the other way — always the + parent, whatever the path names (``molmcp.runtime`` / + ``_session_capability_overlays``). The two rules can only disagree when a + component's ``path`` names a directory, and which one is right there + depends on whether its ``entrypoint`` spells the module relative to that + directory or to the directory above it, which the catalog grammar does not + settle. If a real catalog's provider entrypoint ever fails to import, this + difference is the first thing to check. + + Args: + base: The directory this source's component paths resolve under — + :meth:`ComponentFold.root_for`'s answer. Deliberately not named + ``tree``: :attr:`Checkout.tree` means "where ``harness.toml`` + sits" in this same module, and the two stop being one directory + the moment a catalog declares a ``component_root``. This + function is not told which case it is in and does not need to + be; it takes a base directory and knows nothing about where it + came from. + path: The component's POSIX path, exactly as its catalog authored + it, resolved under *base*. + + Returns: + The directory to import the component from. + """ + candidate = base / path + return candidate if candidate.is_dir() else candidate.parent diff --git a/src/molmcp/harness_install.py b/src/molmcp/harness_install.py new file mode 100644 index 0000000..0cff66b --- /dev/null +++ b/src/molmcp/harness_install.py @@ -0,0 +1,197 @@ +"""``molmcp init`` installs what the *activated* harness commit declares. + +The read half of the harness chain, and the link that was missing from it. +``molmcp config harness set`` registers a source, ``molmcp harness sync`` +publishes its ``HEAD`` and promotes that source's activation pointer, and +:func:`~molmcp.host.place_components` places one file per row — but nothing +turned a *pointer* into those rows, so an operator who had synced a harness +and run ``molmcp init`` got none of it. + +Four obligations, and they are the whole of this module: + +* read each configured source's activation pointer for its ``current`` SHA, + and **skip** a source that has none — a configured source is not a synced + one, and an operator who has not synced yet is not misconfigured; +* load that commit's catalog out of the published tree; +* keep the non-bundle rows, strip the kind's path prefix off each ``path`` + for the host-relative destination, and join ``component_root`` for the + absolute source; +* resolve every row **under its own source's root**, so a multi-source + install never reads one source's components out of another's tree. + +One argument, ``host``, matching :func:`~molmcp.host.install_skill` and +:func:`~molmcp.host.write_adapter` beside it in ``cli._init``: this resolves +*and* places, so composing it costs that function one call. It reads the +configured sources and the cache root itself because ``molmcp init`` takes no +``--config`` flag and has no :class:`~molmcp.config.AppConfig` to be handed. + +**This module stays off the worker stack.** :mod:`molmcp.harness` is the +other reader of these pointers, but it carries +``from .provider_worker.worker import WorkerProvider``, so importing it drags +the whole FastMCP-bearing worker stack into the importing process. +``molmcp init`` mounts no plane and must not pay for one, so the three names +needed here — :class:`~molmcp.components.Activation`, +:class:`~molmcp.components.ImmutableGitStore` and +:func:`~molmcp.components.load_harness_catalog` — are reached in the +stdlib-only :mod:`molmcp.components` leaf that owns them, and the path +spellings this must share with the serve and sync halves come from the light +:mod:`molmcp.harness_paths`. Neither :mod:`molmcp.harness` nor +:mod:`molmcp.provider_worker` may be imported here, however indirectly. +""" + +from __future__ import annotations + +from pathlib import Path + +from .components import ( + KIND_PATH_PREFIX, + Activation, + ComponentSpec, + GitHubTransport, + ImmutableGitStore, + load_harness_catalog, +) +from .config import AppConfig, ConfigurationError +from .harness_paths import SUPPORTED_CAPABILITIES, pointer_path, store_path +from .host import ComponentFile, Host, PlacementReport, place_components +from .runtime import resolved_cache_dir +from .settings import HarnessSource, load_settings + + +def install_harness_components(host: Host) -> PlacementReport: + """Place every component the activated harness commits declare into *host*. + + One pass over the ``harness`` settings list, in the order it names its + sources, then one :func:`~molmcp.host.place_components` call with + everything they declared. A single call rather than one per source + because that function checks every row before it writes the first byte: + folded into one pass, a source whose tree is missing a declared file + leaves nothing behind at all, where a call per source would have + installed its predecessors already. + + Nothing here globs a tree. The catalog is the inventory, so a file + sitting in a published commit that no row names is not a component and + cannot reach a host. + + Args: + host: One of the known hosts, as ``molmcp init`` names it. It is + :func:`~molmcp.host.place_components` that validates it, and that + happens even when nothing was declared. + + Returns: + The report of that one placement run — what was written, what it + replaced, and which rows were refused with which reason. Empty on an + install that configures no harness source, and on one that has + configured sources but has synced none of them: nothing activated is + the ordinary state of a new install, not a broken one. + + Raises: + ConfigurationError: A source's name cannot name a pointer file (see + :func:`~molmcp.harness_paths.pointer_path`), or its pointer + activates a commit that has no published tree. + CatalogError: An activated commit's catalog is malformed, or requires + a capability this build does not support. One bad catalog fails + the install rather than being passed over in favour of its + neighbours — carrying on would install a set the operator did not + select. + ActivationVersionError: A pointer file exists and is not a version-1 + activation record. Raised by + :meth:`~molmcp.components.Activation.bind`; a *missing* file is + not an error, it is the empty record that skips its source. + FileNotFoundError: A catalog declares a file its own tree does not + hold. Raised by :func:`~molmcp.host.place_components` before + anything is written. + """ + settings = load_settings(Path.cwd()) + root = resolved_cache_dir(AppConfig.default(Path.cwd(), settings=settings)) + store = ImmutableGitStore(root=store_path(root), transport=GitHubTransport()) + declared: list[ComponentFile] = [] + for source in settings.harness: + declared.extend(_declared_files(source, root, store)) + return place_components(host, declared) + + +def _declared_files( + source: HarnessSource, root: Path, store: ImmutableGitStore +) -> tuple[ComponentFile, ...]: + """Describe every component one source's activated commit declares. + + The pointer is *read*, never written: staging, promoting and fetching a + commit belong to ``molmcp harness sync``, which was asked to change what + is activated. A source with no pointer file, or with a pointer that + activates nothing, contributes nothing and is not an error, and it does + so per source — the neighbour still yields its own rows. + + Args: + source: One entry of the ``harness`` settings list. + root: The resolved cache root that source's pointer hangs off. + store: The one shared store every source publishes into. + + Returns: + One :class:`~molmcp.host.ComponentFile` per row of that commit's + catalog, in catalog order, every one of them resolved under *this* + source's own base. Bundles yield nothing: they are named groups of + rows rather than files, and the catalog keeps them in a separate + collection. The empty tuple when the source is not activated. + + Raises: + ConfigurationError: The source's name cannot name a pointer file, or + its pointer names a commit with no published tree. The second is + named rather than silently re-fetched: installing a different + commit than the one that was activated is the one outcome nobody + asked for. + """ + pointer = pointer_path(root, source.name) + activation = Activation.bind( + pointer, + store=store, + supported_capabilities=SUPPORTED_CAPABILITIES, + ) + sha = activation.current + if sha is None: + return () + if not store.has(sha): + raise ConfigurationError( + f"the harness source named {source.name!r} is activated at commit " + f"{sha}, which has no published tree under {store_path(root)}. Run " + f"`molmcp harness sync {source.name}` to publish it again, or " + f"delete that source's activation pointer at {pointer}." + ) + tree = store.tree_path(sha) + catalog = load_harness_catalog(tree, sha, SUPPORTED_CAPABILITIES) + base = tree / catalog.component_root if catalog.component_root else tree + return tuple( + _component_file(spec, base) + for spec in catalog.enabled_components(source.enable) + ) + + +def _component_file(spec: ComponentSpec, base: Path) -> ComponentFile: + """Translate one catalog row into the four plain values ``host/`` takes. + + The kind's path prefix is catalog grammar — ``skills/`` says which kind a + row is, which its ``kind`` field already said — so it is stripped here + and the remainder is what the host's own ``skills/`` directory holds. The + prefix is guaranteed present and to have something after it: + :class:`~molmcp.components.ComponentSpec` refuses a path without both. + + Args: + spec: The catalog row, exactly as its catalog declared it. + base: The directory *this* row's source resolves its paths under — + the published tree, or the ``component_root`` inside it. + + Returns: + The description :func:`~molmcp.host.place_components` places. The kind + crosses as a plain string, not as the enum: which kinds have a host + destination is that function's table, and handing it a catalog type + would make it a second reader of catalog grammar. + """ + return ComponentFile( + id=spec.id, + kind=str(spec.kind), + relative=spec.path.removeprefix(KIND_PATH_PREFIX[spec.kind]), + source=base / spec.path, + ) + + +__all__ = ["install_harness_components"] diff --git a/src/molmcp/harness_paths.py b/src/molmcp/harness_paths.py new file mode 100644 index 0000000..d8aedb9 --- /dev/null +++ b/src/molmcp/harness_paths.py @@ -0,0 +1,205 @@ +"""Where a harness commit, its activation, and its checkout live — spelled once. + +Three commands meet on the same files. ``molmcp harness sync`` publishes a +commit and promotes a pointer, ``molmcp serve`` reads that pointer, and +``molmcp init`` reads it again to install what the commit's catalog declares. +A commit published anywhere but :func:`store_path` is one nothing serves, a +pointer written anywhere but :func:`pointer_path` is one nothing reads, and a +checkout one command probes at ``~/harness`` cannot be one another reads at +``./~/harness`` — so each of those answers has exactly one home, and it is +this module. + +:data:`SUPPORTED_CAPABILITIES` rides with them because it travels with them: +every caller that names a pointer also hands that set to +:meth:`~molmcp.components.Activation.bind` and to +:func:`~molmcp.components.load_harness_catalog` in the same breath. Two light +modules for one import would be a split with no seam in it. + +**This module is the light one.** :mod:`molmcp.harness` carries +``from .provider_worker.worker import WorkerProvider``, so importing it drags +the whole FastMCP-bearing worker stack into the importing process. That is a +cost ``molmcp serve`` pays anyway and ``molmcp init`` — which mounts no plane +— must not, which is why these four names live below it rather than in it: +:mod:`molmcp.harness_install` reaches them without inheriting the worker +stack, while :mod:`molmcp.harness` and :mod:`molmcp.harness_sync` reach the +very same objects. Nothing beyond :mod:`molmcp.config` and +:mod:`molmcp.settings` may be imported here, or the shelter this module +exists to give is gone. +""" + +from __future__ import annotations + +import os +from pathlib import Path + +from .config import ConfigurationError +from .settings import HarnessSource + +#: Capability tokens this runtime can honor, named once here and passed as +#: this object to :meth:`Activation.bind` and to every catalog load. +#: +#: A *harness* is a git repository holding the user's own agent tooling — +#: skills, agents, rules, provider planes, discovery overlays — that this +#: install can be pointed at. Its ``harness.toml`` *catalog* declares those +#: pieces, and the catalog (and each named bundle inside it) may list +#: *capability tokens*: machinery a piece needs from whatever process loads +#: it. The two this build honors are ``provider-sdk``, the public +#: :mod:`molmcp.provider_sdk` a checkout plane is written against, and +#: ``harness-catalog``, the catalog format read by +#: :func:`~molmcp.components.load_harness_catalog`. +#: +#: This set is deliberately not ``molmcp.components.ALLOWED_REQUIRES``. That +#: set is what a harness catalog is *allowed to declare* — the grammar. This +#: one is what this process can *deliver* — eligibility. They happen to hold +#: the same two tokens today; aliasing them would make a token added to the +#: grammar tomorrow claim runtime support that nothing here implements. +SUPPORTED_CAPABILITIES = frozenset({"provider-sdk", "harness-catalog"}) + +#: The shared store's directory name under the resolved cache root. Spelled +#: once here and read through :func:`store_path`; see that function for why it +#: is not a literal at its call sites. +_STORE_DIR_NAME = "harness" + +#: Source names :func:`pointer_path` refuses outright, kept for symmetry with +#: :data:`molmcp.components.store._RESERVED_SHA_KEYS` rather than because +#: either one escapes a directory — see that function's docstring. +_RESERVED_SOURCE_NAMES = frozenset({".", ".."}) + + +def pointer_path(root: Path, name: str) -> Path: + """Name the activation pointer file of one harness source. + + Each source owns ``/harness..pointer``, a direct child of the + cache root and a sibling of the one shared store at ``/harness``. + + The name is guarded here rather than on + :class:`~molmcp.settings.HarnessSource`, because this is the only place + that knows the name is about to become path *structure* instead of a + label: that class governs it as "non-empty and whitespace-free" on + purpose, so an operator who may name an index source ``MolCrafts`` may + name a harness source ``MolCrafts``, and ``molmcp config`` must keep + working on a settings file this function refuses. + + The guard is :meth:`ImmutableGitStore._sha_dir`'s, and **which half of it + is load-bearing is worth stating**, so that nobody later "simplifies" it + by dropping the half that matters. ``.`` and ``..`` are refused for + symmetry with that method's reserved set, **not** because they traverse: + interpolated into ``harness.{name}.pointer`` neither is a path segment at + all — ``harness....pointer`` is one ordinary filename inside *root*. The + separator, absolute-path and empty checks are the ones that close the + hole, since ``a/b`` and ``../../evil`` do turn the name into structure + and would write outside the cache root. + + Nothing is created, and nothing is created on the way to a refusal: this + function computes a path and never touches the filesystem. + + Args: + root: The resolved cache root the store already hangs off. + name: The harness source's name, as the ``harness`` settings list + spells it. + + Returns: + The pointer file for that source. + + Raises: + ConfigurationError: The name cannot be one path segment — it is + empty, reserved, absolute, or contains a path separator. The + message names it with ``repr``, this repo's register for a + rejected value and the only form that can name the empty string + at all. + """ + if ( + not name + or name in _RESERVED_SOURCE_NAMES + or Path(name).is_absolute() + or os.sep in name + or "/" in name + or "\\" in name + or (os.altsep is not None and os.altsep in name) + ): + raise ConfigurationError( + f"the harness source named {name!r} cannot name an activation " + f"pointer file: a source name must be a single path segment, so " + f"it may not be empty, `.`, `..`, absolute, or contain a path " + f"separator. Rename that entry of the `harness` list in your " + f"settings file." + ) + return root / f"harness.{name}.pointer" + + +def store_path(root: Path) -> Path: + """Name the one shared store every harness source publishes into. + + Every source's commits land under ``/harness``, a sibling of the + per-source pointer files :func:`pointer_path` names. One directory, not + one per source: :class:`~molmcp.components.ImmutableGitStore` keys a + commit on its SHA alone, so a second root would buy no isolation and + would strand every already-published tree. + + It is a function rather than a literal spelled at each call site because + it has three callers that must agree exactly — ``molmcp harness sync``, + which publishes into it, and the two readers, :mod:`molmcp.harness` at + serve time and :mod:`molmcp.harness_install` at ``molmcp init`` time. A + sync writing anywhere else would leave both readers unable to find the + commit that was just activated, and the failure would look like a corrupt + pointer rather than like a typo. + + Nothing is created here: this computes a path and never touches the + filesystem. + + Args: + root: The resolved cache root. + + Returns: + The shared store directory under *root*. + """ + return root / _STORE_DIR_NAME + + +def local_checkout_path(source: HarnessSource) -> Path: + """Name the directory one local harness entry's ``path`` points at. + + The string an operator stores is not always the directory to read. + :func:`~molmcp.harness.assert_servable` accepts ``~/harness`` — home is + the same directory in every session, so that entry names one checkout + rather than a different one per client — which makes the home-relative + spelling the one servable ``path`` that must be expanded before anything + opens it. Handed to a transport as written, ``~/harness`` is an ordinary + two-segment relative path read against whatever working directory the + client that launched the process happened to stand in. + + A function rather than an ``expanduser()`` at each call site, for the + reason :func:`store_path` is one: it has two callers that must agree + exactly — the servability check in :mod:`molmcp.harness` and ``molmcp + harness sync``'s choice of transport root. A checkout ``molmcp serve`` + probes at one location cannot be one ``molmcp harness sync`` clones from + another, which is the failure two spellings drift into. + + **Only ``~`` is expanded.** :meth:`Path.resolve` would turn the + working-directory-relative spellings + :func:`~molmcp.harness.assert_servable` exists to refuse into absolute + paths, so the refusal would stop firing; it would also normalise the + operator's stored string — possibly authored on another machine — into + this machine's answer, which is the bug in the same family. Nothing is + created and nothing is read here: this computes a path and never touches + the filesystem. + + Args: + source: One entry of the ``harness`` settings list, whose ``path`` + the caller has already found non-empty. An entry naming a GitHub + coordinate has no local checkout at all, and its empty ``path`` + would come back as the working directory rather than as nothing. + + Returns: + The directory that entry's ``path`` names, with a leading ``~`` + expanded to this session's home. + """ + return Path(source.path).expanduser() + + +__all__ = [ + "SUPPORTED_CAPABILITIES", + "local_checkout_path", + "pointer_path", + "store_path", +] diff --git a/src/molmcp/harness_sync.py b/src/molmcp/harness_sync.py new file mode 100644 index 0000000..952d3b4 --- /dev/null +++ b/src/molmcp/harness_sync.py @@ -0,0 +1,610 @@ +"""``molmcp harness sync`` and ``rollback``: the two verbs that move a pointer. + +``molmcp config harness set`` writes a coordinate and ``molmcp serve`` reads an +activation pointer. This module is what runs in between — resolve the named +source's ref to a commit, publish that commit into the shared store, activate +it in that source's own pointer — and it is the first production caller of +:meth:`~molmcp.components.ImmutableGitStore.publish`, +:meth:`~molmcp.components.Activation.stage` and +:meth:`~molmcp.components.Activation.promote`. + +:func:`rollback_source` is the other direction along that same pointer, and the +first production caller of :meth:`~molmcp.components.Activation.rollback`: a +sync records the SHA it displaced in ``previous`` for exactly one reason, and +this is that reason. It sits beside the sync rather than in a sibling module +because both verbs address a source by the same operator-chosen label out of +the same settings list, so both owe an unknown name the same sentence. +:func:`_named` and :func:`_bind` are theirs jointly, and a second module could +reach them only by importing a private name or by keeping a second copy of a +message whose whole value is that it does not depend on which verb was typed. + +Its own module rather than more of :mod:`molmcp.harness`, whose stated identity +is that serving "is a read of the activation pointers and of each checkout's +``harness.toml`` — never a fetch, never a write". Fetching and writing are this +module's whole job, so folding them in there would make that sentence false. +What the two share is spelled once and imported: :func:`~molmcp.harness. +assert_servable` (which entries this install may reach), and from the light +:mod:`molmcp.harness_paths` leaf that ``molmcp init`` reads too, +:func:`~molmcp.harness_paths.local_checkout_path` (which directory a local +entry's ``path`` names), :func:`~molmcp.harness_paths.store_path` and +:func:`~molmcp.harness_paths.pointer_path` (where a commit and its activation +land). + +**Transport is chosen by the source's shape, never by a flag.** +:class:`~molmcp.settings.HarnessSource` already refuses an entry carrying both +a ``path`` and a coordinate, so the entry itself is a total answer to "where +does this come from". A ``--local`` flag would be a second answer, and two +answers to one question is how an install ends up fetching from a repository +nobody named. + +**No network is opened here.** Both transports are constructed here and neither +is spoken to except through :class:`~molmcp.components.GitTransport`; the local +one shells out to ``git`` in a checkout on disk and opens no socket at all. +:func:`rollback_source` opens nothing at all: it names a transport only because +:meth:`~molmcp.components.Activation.bind` requires a store and a store requires +one, exactly as the two read-only callers in :mod:`molmcp.harness` and +:mod:`molmcp.harness_install` do, and it never speaks to it. +""" + +from __future__ import annotations + +import os +from collections.abc import Sequence +from dataclasses import dataclass +from pathlib import Path + +from .components import ( + Activation, + CatalogError, + GitHubTransport, + GitTransport, + ImmutableGitStore, + LocalGitTransport, + load_harness_catalog, +) +from .components.activate import ( + ActivationVersionError, + IneligibleShaError, + NothingToRollbackError, +) +from .components.store import StoreError +from .config import AppConfig, ConfigurationError +from .harness import assert_servable +from .harness_paths import ( + SUPPORTED_CAPABILITIES, + local_checkout_path, + pointer_path, + store_path, +) +from .runtime import resolved_cache_dir +from .settings import ( + HarnessSource, + SettingsError, + load_settings, + match_harness_source, + read_settings_file, + set_harness_source, +) + + +@dataclass(frozen=True, slots=True) +class SyncReport: + """What one :func:`sync_source` call did, for the caller to print. + + Attributes: + source: Name of the harness source that was synced. + sha: Commit the source's ref resolved to, and the one now activated. + tree: Published catalog root for that commit, under + :func:`~molmcp.harness_paths.store_path`. + pointer: Activation pointer file this source owns. + promoted: ``True`` when the pointer moved, ``False`` when *sha* was + already the activated commit and nothing was staged. The + distinction is not cosmetic: promoting a commit that is already + current would overwrite ``previous`` — the one SHA + :meth:`~molmcp.components.Activation.rollback` returns to — with + the SHA that is already current, and the install would silently + lose its way back. + """ + + source: str + sha: str + tree: Path + pointer: Path + promoted: bool + + +@dataclass(frozen=True, slots=True) +class RollbackReport: + """What one :func:`rollback_source` call did, for the caller to print. + + No published tree is named, unlike :class:`SyncReport`. A rollback reads + no tree at all, and :meth:`~molmcp.components.ImmutableGitStore.tree_path` + raises on a commit whose directory has since been pruned — a field nothing + needed would have turned a pointer move into a traceback. + + Attributes: + source: Name of the harness source whose pointer moved. + sha: Commit now activated — the one the last sync displaced and + recorded as ``previous``. + pointer: Activation pointer file this source owns, now naming *sha*. + """ + + source: str + sha: str + pointer: Path + + +def sync_source(config: AppConfig, name: str) -> SyncReport: + """Fetch, publish and activate the commit one named harness source is at. + + The four steps, in the one order that leaves nothing half-done: resolve + the ref to a commit, publish that commit's tree, then — only if it is not + already the activated one — stage it and promote it. Publishing before + reading the pointer is deliberate: publishing a commit that is already + published is a no-op that touches no directory, and doing it first repairs + an install whose store was pruned out from under a still-valid pointer. + + Args: + config: **Already-resolved** application configuration. The caller + resolves it so that the store and the pointer land under the very + same cache root ``molmcp serve`` reads, and so this function never + has to decide where a cache lives. + name: The harness source to sync, matched exactly against the ``name`` + of an entry in the ``harness`` settings list. + + Returns: + What was done, including whether the pointer actually moved. + + Raises: + ConfigurationError: No entry is named *name* (the message lists the + ones that are configured); the entry names no origin this install + can reach; the source's pointer file is not a readable activation + record; the store refuses the commit; or the commit's + ``harness.toml`` is one this build cannot serve. + GitError: The ref did not resolve, or the archive could not be + fetched. Raised by the transport and deliberately not reworded — + its message is the only place the reason is written down. + """ + source = _named(_loaded_harness(), name) + assert_servable(source) + + root = resolved_cache_dir(config) + pointer = pointer_path(root, source.name) + transport = _transport(source) + sha = transport.resolve_commit(source.owner, source.repo, source.ref or None) + store = ImmutableGitStore(root=store_path(root), transport=transport) + tree = _publish(store, source, sha) + try: + catalog = load_harness_catalog(tree, sha, SUPPORTED_CAPABILITIES) + catalog.enabled_components(source.enable) + except CatalogError as exc: + raise ConfigurationError( + f"the harness source named {source.name!r} cannot be served: {exc}" + ) from exc + + activation = _bind(pointer, store, source) + if activation.current == sha: + return SyncReport( + source=source.name, + sha=sha, + tree=tree, + pointer=pointer, + promoted=False, + ) + _stage_and_promote(activation, source, sha, tree) + return SyncReport( + source=source.name, + sha=sha, + tree=tree, + pointer=pointer, + promoted=True, + ) + + +def rollback_source(config: AppConfig, name: str) -> RollbackReport: + """Activate the commit this source's last sync displaced. + + Two steps, and neither of them fetches: bind the named source's activation + pointer, then move it back one level. ``previous`` is what + :meth:`~molmcp.components.Activation.promote` recorded when it activated a + commit over the one that was current, and this is the only thing that + reads it. + + **One level, not a toggle.** :meth:`~molmcp.components.Activation.rollback` + clears ``previous`` as it restores it, so the record left behind names a + current commit and no way back: a second call refuses exactly as a + never-synced source does. The way *forward* to the newer commit is + ``molmcp harness sync``, and it re-fetches nothing, because a rollback + prunes nothing and that commit's tree is still published. + + **The entry's origin is never consulted.** + :func:`~molmcp.harness.assert_servable` is deliberately not called and no + checkout is opened, because a rollback reaches no origin. Refusing an entry + here for an origin this install can no longer reach would strand precisely + the operator this verb exists for — the one whose checkout has since moved + and whose good commit is still sitting published in the store. + + Args: + config: **Already-resolved** application configuration, for the reason + :func:`sync_source` takes one: the pointer this moves has to be + the file under the very same cache root ``molmcp serve`` reads. + name: The harness source to roll back, matched exactly against the + ``name`` of an entry in the ``harness`` settings list. + + Returns: + The source, the commit now activated, and the pointer that says so. + + Raises: + ConfigurationError: No entry is named *name* (the message lists the + ones that are configured); the source's pointer file is not a + readable activation record; or that record names no previous + commit, which is the state of a source synced once and of one + already rolled back alike. Nothing is written on any of those + paths — a source that was never synced still has no pointer file + afterwards, since one written here is one ``molmcp serve`` and + ``molmcp init`` would then have to read. + """ + source = _named(_loaded_harness(), name) + root = resolved_cache_dir(config) + pointer = pointer_path(root, source.name) + store = ImmutableGitStore(root=store_path(root), transport=GitHubTransport()) + activation = _bind(pointer, store, source) + return RollbackReport( + source=source.name, + sha=_roll_back(activation, source, pointer), + pointer=pointer, + ) + + +def relocate_pointer( + config: AppConfig, + settings_path: Path, + *, + locator: str, + name: str, + enable: Sequence[str] = (), + disable: Sequence[str] = (), +) -> None: + """Rename one harness source and move its activation pointer with it. + + :func:`~molmcp.settings.set_harness_source` is the settings write; + :func:`~molmcp.harness_paths.pointer_path` still names the file from + the alias. This function is the one place an alias change also moves + that file. The target pointer must not already exist unless it *is* + the source file: refusing first is what leaves the settings file + unchanged. No pointer on disk is settings-only. + + Args: + config: Already-resolved application configuration, so the + pointer paths land under the same cache root ``molmcp serve`` + reads. + settings_path: The settings file that holds the entry. + locator: Origin as the operator wrote it; identity is its origin + key. + name: The new alias. + enable: Passed through to + :func:`~molmcp.settings.set_harness_source`. + disable: Passed through to + :func:`~molmcp.settings.set_harness_source`. + + Raises: + ConfigurationError: The new pointer file already exists under a + different path, or the settings layer refuses the rename. + """ + matched = match_harness_source(_file_harness(settings_path), locator) + old_pointer: Path | None = None + new_pointer: Path | None = None + same_pointer = True + if matched is not None: + root = resolved_cache_dir(config) + old_pointer = pointer_path(root, matched.name) + new_pointer = pointer_path(root, name) + same_pointer = old_pointer == new_pointer or ( + old_pointer.exists() + and new_pointer.exists() + and os.path.samefile(old_pointer, new_pointer) + ) + if new_pointer.exists() and not same_pointer: + raise ConfigurationError( + f"cannot rename harness source {matched.name!r} to {name!r}: " + f"the activation pointer {new_pointer} already exists" + ) + _set_harness_source( + settings_path, + locator, + alias=name, + enable=enable, + disable=disable, + ) + if ( + old_pointer is not None + and new_pointer is not None + and old_pointer.exists() + and not same_pointer + ): + os.replace(old_pointer, new_pointer) + + +def _named(sources: Sequence[HarnessSource], name: str) -> HarnessSource: + """Select the entry matching *name* as an alias or a locator spelling. + + Naming the typo is only half the message. A source is addressed by an + operator-chosen label or by any spelling of its origin, so "unknown + source" on its own leaves them to go and read the settings file to + find out what they should have typed. + + Args: + sources: The ``harness`` list as the settings files resolved it. + name: An alias, or any accepted locator spelling of an origin. + + Returns: + The one matching entry. + + Raises: + ConfigurationError: No entry matches *name*. + """ + matched = match_harness_source(sources, name) + if matched is not None: + return matched + configured = ", ".join(repr(source.name) for source in sources) or "(none)" + raise ConfigurationError( + f"no harness source is named {name!r}. This install configures: " + f"{configured}. Sync one of those, or add the entry first with " + f"`molmcp config harness set {name}`." + ) + + +def _loaded_harness() -> tuple[HarnessSource, ...]: + """The resolved ``harness`` list, with settings failures as config errors.""" + try: + return load_settings(Path.cwd()).harness + except SettingsError as exc: + raise ConfigurationError(str(exc)) from exc + + +def _file_harness(path: Path) -> tuple[HarnessSource, ...]: + """The ``harness`` entries stored in one settings file.""" + try: + raw = read_settings_file(path) + except SettingsError as exc: + raise ConfigurationError(str(exc)) from exc + entries = raw.get("harness", []) + if not isinstance(entries, list): + return () + return tuple(HarnessSource(**entry) for entry in entries if isinstance(entry, dict)) + + +def _set_harness_source( + path: Path, + locator: str, + *, + alias: str | None, + enable: Sequence[str], + disable: Sequence[str], +) -> None: + """Call :func:`set_harness_source`, mapping a settings refusal up.""" + try: + set_harness_source(path, locator, alias=alias, enable=enable, disable=disable) + except SettingsError as exc: + raise ConfigurationError(str(exc)) from exc + + +def _transport(source: HarnessSource) -> GitTransport: + """Build the transport this entry's *shape* calls for. + + A local locator is a checkout on disk and gets + :class:`~molmcp.components.LocalGitTransport` rooted at + :func:`~molmcp.harness_paths.local_checkout_path`; a GitHub locator + gets :class:`~molmcp.components.GitHubTransport`. No flag participates + — see the module docstring. + + ``assert_servable`` has already run, and what that buys is narrower than + "the path is ready to use": a local locator names a real checkout + **once expanded**. The expansion is still this function's to do, and it + is done by calling :func:`~molmcp.harness_paths.local_checkout_path` + rather than by a second ``expanduser()`` here — a home-relative + ``~/harness`` would otherwise root this transport at a *literal* ``~`` + directory under whatever working directory the client that launched + this process stood in. A GitHub locator has an empty ``path``; its + ``owner`` / ``repo`` / ``ref`` are derived, and ``ref or None`` is what + :meth:`~molmcp.components.GitHubTransport.resolve_commit` is handed. + + Args: + source: The entry to build a transport for. + + Returns: + The transport for that origin. No token is passed to the GitHub one: + a credential belongs in the environment of whatever reads it, and + nothing in this module reads the environment. + """ + if source.is_local: + return LocalGitTransport(local_checkout_path(source)) + return GitHubTransport() + + +def _publish(store: ImmutableGitStore, source: HarnessSource, sha: str) -> Path: + """Install *sha*'s tree in the shared store and return its catalog root. + + Provenance is the entry's own ``owner`` and ``repo``, passed through + unchanged — including the two empty strings a local entry has. Inventing + a coordinate for a local source (its path, say) would make two clones of + one repository claim one SHA under two owners, and + :class:`~molmcp.components.ShaConflictError` would then refuse the second + sync of a commit whose tree is byte-for-byte the one already published. + + Args: + store: The shared store, already rooted at :func:`store_path`. + source: The entry being synced, read for provenance only. + sha: The commit to publish. + + Returns: + The published catalog root for *sha*. + + Raises: + ConfigurationError: The store refused the commit — most reachably, + *sha* is already published under a different repository, which + happens when an entry is moved from one origin to another. + """ + try: + return store.publish(sha, owner=source.owner, repo=source.repo) + except StoreError as exc: + raise ConfigurationError( + f"the harness source named {source.name!r} resolved to commit " + f"{sha}, which this install's harness store will not publish: " + f"{exc}. Nothing was activated, so the commit that was serving " + f"still is." + ) from exc + + +def _bind(pointer: Path, store: ImmutableGitStore, source: HarnessSource) -> Activation: + """Bind this source's activation pointer, naming the file if it is broken. + + Args: + pointer: The source's own ``harness..pointer`` file. A missing + one is not an error — it binds an empty record, which is the + never-synced install. + store: The shared store the activation checks eligibility against. + source: The entry being synced or rolled back, named in the failure + message. + + Returns: + The bound activation. + + Raises: + ConfigurationError: The file exists and is not a version-1 activation + record. + """ + try: + return Activation.bind( + pointer, + store=store, + supported_capabilities=SUPPORTED_CAPABILITIES, + ) + except ActivationVersionError as exc: + raise ConfigurationError( + f"the activation pointer of the harness source named " + f"{source.name!r} is not a readable activation record: {exc}. " + f"Delete {pointer} and sync again — a pointer holds names, not " + f"trees, so nothing published is lost with it." + ) from exc + + +def _stage_and_promote( + activation: Activation, source: HarnessSource, sha: str, tree: Path +) -> None: + """Stage *sha* past the eligibility gate, then make it the current commit. + + Both steps or neither: ``stage`` is what reads the commit's + ``harness.toml`` and refuses one this build cannot honor, and ``promote`` + is what a served process would see. A staged SHA left unpromoted is a SHA + nothing serves, which is indistinguishable from a sync that never ran. + + Args: + activation: The bound pointer to move. + source: The entry being synced, named in the failure message. + sha: The commit to activate. + tree: That commit's published catalog root, named in the failure + message so there is a directory to go and look at. + + Raises: + ConfigurationError: The commit's catalog is malformed, or requires a + capability this build does not provide. The pointer is left + exactly as it was. + """ + try: + activation.stage(sha) + except IneligibleShaError as exc: + # The catalog's filename is deliberately absent from this sentence. + # `components/catalog.py` is the one module allowed to resolve it + # (`tests/test_harness_catalog_fixture.py` enforces that), so this + # names the published tree and lets the operator find the file in it. + provided = ", ".join(sorted(SUPPORTED_CAPABILITIES)) + raise ConfigurationError( + f"the harness source named {source.name!r} resolved to commit " + f"{sha}, whose catalog this build cannot serve: it is malformed, " + f"or it requires a capability beyond the ones this molmcp " + f"provides ({provided}). The tree is published at {tree}. Nothing " + f"was activated, so the commit that was serving still is." + ) from exc + activation.promote() + + +def _roll_back(activation: Activation, source: HarnessSource, pointer: Path) -> str: + """Move *activation* back one level and return the commit now activated. + + The refusal is raised from two sites and the two are not redundant. The + guard runs before anything is written, and it is the one an operator hits: + a verb that reported "nothing to roll back" over a record it had already + replaced would leave the install activating nothing at all, which is worse + than the state it refused. The handler answers the same condition as seen + by :meth:`~molmcp.components.Activation.rollback`'s own reload of the file, + which is what another process moving the pointer in between looks like. + Both spell one sentence, because it is one condition. + + Converting rather than leaving it to ``cli.main``'s funnel is the choice + here: ``NothingToRollbackError`` is an ``ActivationError``, which is a + plain ``Exception``, so it is caught by none of the types that funnel + registers and would otherwise reach the operator as a traceback. The other + way to close that is to register ``ActivationError`` there, but the raw + message is ``nothing to rollback`` — it names neither the source nor the + way forward, so it is not a sentence a CLI can hand over. That is why + ``IneligibleShaError``, ``StoreError`` and ``ActivationVersionError`` are + converted in this module too, and the opposite of ``GitError``, whose own + message already names the ref or the checkout git could not answer for. + + Args: + activation: The bound pointer to move. + source: The entry being rolled back, named in the failure message. + pointer: That source's pointer file, named in the failure message so + there is a file to go and look at. + + Returns: + The commit that is activated once the pointer has moved: the one + ``previous`` named. + + Raises: + ConfigurationError: The record names no previous commit. The pointer + file is left exactly as it was, and a missing one is not created. + """ + previous = activation.previous + if previous is None: + raise _nothing_to_roll_back(source, pointer) + try: + activation.rollback() + except NothingToRollbackError as exc: + raise _nothing_to_roll_back(source, pointer) from exc + return previous + + +def _nothing_to_roll_back(source: HarnessSource, pointer: Path) -> ConfigurationError: + """Build the refusal for an activation record with no previous commit. + + Returned rather than raised so that both sites in :func:`_roll_back` hand + the operator the same sentence without a second copy of it. + + Args: + source: The entry that was to be rolled back. + pointer: That source's activation pointer file. + + Returns: + The error to raise. + """ + return ConfigurationError( + f"the harness source named {source.name!r} has no commit to roll back " + f"to: its activation pointer records no previous commit. That is the " + f"state of a source synced only once, and of one already rolled back " + f"— rollback clears the previous commit as it restores it, so it goes " + f"back one level rather than toggling between two. To move forward " + f"again, run `molmcp harness sync {source.name}`; the commit it " + f"activates is still published, so nothing is re-fetched. Nothing was " + f"written to {pointer}." + ) + + +__all__ = [ + "RollbackReport", + "SyncReport", + "relocate_pointer", + "rollback_source", + "sync_source", +] diff --git a/src/molmcp/host/__init__.py b/src/molmcp/host/__init__.py new file mode 100644 index 0000000..140a923 --- /dev/null +++ b/src/molmcp/host/__init__.py @@ -0,0 +1,77 @@ +"""Host adapter layer: where ``molmcp init`` writes on each known host. + +A *host* is the AI client a user runs — a desktop app or a terminal agent. +:data:`HOSTS` names the ones molmcp knows, and each keeps its configuration in +its own directory under the user's home. ``molmcp init `` fills that +tree with six kinds of file: + +* the **MCP JSON** — MCP (Model Context Protocol) is the wire protocol an AI + client uses to call tools, and this file is the client's list of servers to + launch; +* the **usage skill**, also called the *constitution* — the managed + ``SKILL.md`` that teaches the agent how to drive molmcp; +* the **adapter** ``molmcp-adapter.md`` — a short pointer file saying where + the others live, carrying no skill, agent, or rule body of its own; +* the **daily bundle** — extra skills for ordinary use, copied in beside the + usage skill; +* the **dev bundle** — the harness a molmcp contributor uses: full bodies + under ``molmcp-dev/`` plus one-line slash-command stubs under ``commands/``; +* the **catalog components** — the skills, agents, and rules a harness + catalog declares, placed one file at a time from an activated commit tree. + +The daily and dev bundles come from a *checkout*: a directory the caller +passes in explicitly. Nothing here goes looking for one. Catalog components +arrive the same way, already resolved: one +:class:`~molmcp.host.place.ComponentFile` per file, described in stdlib types +only, so this package never learns what a catalog is. + +This package re-exports the public surface of :mod:`molmcp.host.layout`, the +single host path table, of :mod:`molmcp.host.install`, the write primitives +that fill it, and of :mod:`molmcp.host.place`, which installs catalog +components into it. It imports the standard library only, so +``client_config`` can read it without an import cycle. +""" + +from .install import ( + ADAPTER_TEXT, + EXTRA_SKILLS, + install_extra_skills, + install_skill, + write_adapter, +) +from .layout import ( + HOSTS, + SKILL_NAME, + Host, + HostLayout, + default_skill_dir, + default_write_path, + layout_for, +) +from .place import ( + SKIP_MANAGED_USAGE_SKILL, + SKIP_NO_HOST_DESTINATION, + ComponentFile, + PlacementReport, + place_components, +) + +__all__ = [ + "ADAPTER_TEXT", + "EXTRA_SKILLS", + "HOSTS", + "SKILL_NAME", + "SKIP_MANAGED_USAGE_SKILL", + "SKIP_NO_HOST_DESTINATION", + "ComponentFile", + "Host", + "HostLayout", + "PlacementReport", + "default_skill_dir", + "default_write_path", + "install_extra_skills", + "install_skill", + "layout_for", + "place_components", + "write_adapter", +] diff --git a/src/molmcp/host/install.py b/src/molmcp/host/install.py new file mode 100644 index 0000000..d6ee1eb --- /dev/null +++ b/src/molmcp/host/install.py @@ -0,0 +1,194 @@ +"""Write primitives for ``molmcp init``: usage skill, daily, adapter, dev. + +Each function here writes exactly one kind of thing and returns what it wrote. +There is deliberately no "do it all" facade: :mod:`molmcp.cli` composes these +in order, so no primitive can quietly grow a second destination. +:mod:`molmcp.host` introduces the vocabulary used below — host, usage skill +(constitution), adapter, daily bundle, dev bundle, checkout. + +Two backends supply content. The *packaged* backend reads the usage +constitution and the extra skills in :data:`EXTRA_SKILLS` from data files +shipped inside the installed distribution (the ``molmcp.skill`` package) +and always applies. The *checkout* backend is a directory the caller +passes in, holding the daily and dev bundles; when the caller passes none, +every bundle primitive is a no-op that creates no empty directory. +:func:`resolve_bundle_source` is the only place that choice is +interpreted — nothing here probes the working directory, a git root, a +sibling checkout, or an environment variable. + +Validation of the host name happens before that no-op check, so an unknown +host still raises even when there is no checkout to materialize. + +This module is Layer 2 and imports the standard library only (plus +``importlib.resources`` reading ``molmcp.skill``). Nothing under +``molmcp.host`` may import ``molmcp.client_config``, ``molmcp.cli``, +``molmcp.server``, ``molmcp.providers``, or ``molmcp.discovery``. +""" + +from __future__ import annotations + +from importlib.resources import files +from pathlib import Path + +from .layout import Host, layout_for, remap_frontmatter + +EXTRA_SKILLS: tuple[str, ...] = ("molexp-plan",) +"""Packaged skills ``molmcp init`` writes beside the usage constitution. + +The constitution is :func:`install_skill` and this tuple is +:func:`install_extra_skills`. A catalog row may overlay an extra skill; +it cannot take the constitution's name. +""" + +ADAPTER_TEXT = """# molmcp adapter + +Wired by `molmcp init`. This file is a pointer, not a constitution. + +- Usage skill: `molcrafts` (auto-loaded). Do not edit the managed SKILL.md. +- MCP: one `molcrafts` server from `molmcp serve`. +- Catalog components: this host's `skills/`, `agents/`, and `rules/` directories. + +Do not copy skill, agent, or rule bodies into this file. +""" +"""Body of every host's ``molmcp-adapter.md``, byte-identical everywhere. + +It is a pointer to where the real bodies live, so it carries no skill, agent, +or rule text, no timestamp, no home path, and no content hash. Every host in +:data:`~molmcp.host.layout.HOSTS` receives these exact bytes, so two machines +wired by the same molmcp version hold the same file. +""" + + +def _home_path(parts: tuple[str, ...]) -> Path: + """Resolve a layout path tuple against the current home directory.""" + return Path.home().joinpath(*parts) + + +def _write(dest: Path, text: str) -> Path: + """Create *dest*'s parent, write *text* as UTF-8 LF, and return *dest*. + + ``newline="\\n"`` is load-bearing: the default on Windows is ``\\r\\n``, + which would make the adapter and the usage skill differ by host. + """ + dest.parent.mkdir(parents=True, exist_ok=True) + dest.write_text(text, encoding="utf-8", newline="\n") + return dest + + +def _usage_skill_file() -> Path: + """Locate the packaged usage constitution ``SKILL.md``. + + The lookup happens here and nowhere else, so a checkout and an installed + wheel name the same file: package data puts ``SKILL.md`` beside + ``molmcp/skill/__init__.py`` in both, leaving no second location to fall + back to. + + Returns: + Path of the ``SKILL.md`` shipped inside :mod:`molmcp.skill`. + """ + return Path(str(files("molmcp.skill") / "SKILL.md")) + + +def install_skill(host: Host) -> Path: + """Overwrite the managed usage skill for *host*. + + The packaged ``SKILL.md`` is *copied*, not re-rendered from a template. + Copying gives a checkout and a PyPI wheel one path — package data places + the same file beside :mod:`molmcp.skill` either way — so the constitution + lands byte-identical, mode and modification time included, and there is + no rendering step that could drift from the file it claims to reproduce. + + Only the usage constitution is written: extra packaged skills, the + adapter pointer, and the daily bundle have their own primitives. + + Args: + host: One of the known hosts. + + Returns: + The ``SKILL.md`` path written. + + Raises: + ValueError: If *host* is not a known host. + OSError: If the packaged ``SKILL.md`` cannot be read; that is a + broken installation, which :func:`molmcp.cli.main` already + reports as a message rather than a traceback. + """ + skill_dir = _home_path(layout_for(host).skill_dir) + skill_dir.mkdir(parents=True, exist_ok=True) + dest = skill_dir / "SKILL.md" + text = _usage_skill_file().read_text(encoding="utf-8") + return _write(dest, remap_frontmatter(text, host)) + + +def _extra_skill_file(name: str) -> Path: + """Locate one packaged extra skill ``SKILL.md``. + + Extra skills live in a subdirectory of :mod:`molmcp.skill` named after + the skill, so the constitution at the package root stays the one file + :func:`install_skill` copies. + + Args: + name: A member of :data:`EXTRA_SKILLS`. + + Returns: + Path of that skill's ``SKILL.md`` inside :mod:`molmcp.skill`. + + Raises: + ValueError: If *name* is not a packaged extra skill. + """ + if name not in EXTRA_SKILLS: + raise ValueError(f"unknown skill {name!r}; known: {', '.join(EXTRA_SKILLS)}") + return Path(str(files("molmcp.skill") / name / "SKILL.md")) + + +def install_extra_skills(host: Host) -> tuple[Path, ...]: + """Overwrite every packaged extra skill for *host*. + + Writes beside the usage constitution, never into its directory. Each + body is remapped the same way :func:`install_skill` remaps the + constitution, so a grok install drops ``metadata:`` here too. + + Args: + host: One of the known hosts. + + Returns: + The ``SKILL.md`` paths written, in :data:`EXTRA_SKILLS` order. + + Raises: + ValueError: If *host* is not a known host. + OSError: If a packaged extra ``SKILL.md`` cannot be read. + """ + skills_root = _home_path(layout_for(host).skill_dir[:-1]) + written: list[Path] = [] + for name in EXTRA_SKILLS: + dest = skills_root / name / "SKILL.md" + text = _extra_skill_file(name).read_text(encoding="utf-8") + written.append(_write(dest, remap_frontmatter(text, host))) + return tuple(written) + + +def write_adapter(host: Host) -> Path: + """Write *host*'s stable pointer file ``molmcp-adapter.md``. + + The body is :data:`ADAPTER_TEXT` verbatim, so re-running ``molmcp init`` + on the same version is a no-diff write. + + Args: + host: One of the known hosts. + + Returns: + The adapter path written. + + Raises: + ValueError: If *host* is not a known host. + """ + return _write(_home_path(layout_for(host).adapter), ADAPTER_TEXT) + + +__all__ = [ + "ADAPTER_TEXT", + "EXTRA_SKILLS", + "install_extra_skills", + "install_skill", + "write_adapter", +] diff --git a/src/molmcp/host/layout.py b/src/molmcp/host/layout.py new file mode 100644 index 0000000..10a392a --- /dev/null +++ b/src/molmcp/host/layout.py @@ -0,0 +1,240 @@ +"""The single host path table and its thin readers. + +A *host* is the AI client ``molmcp init`` wires — a desktop app or a terminal +agent; :mod:`molmcp.host` introduces what gets written into it. One record per +host says where every file ``molmcp init`` may write belongs, so no second +table can drift from it. Values are path tuples relative to ``Path.home()``; +home is resolved only when a caller asks for a concrete +:class:`~pathlib.Path`, and no environment variable selects a destination. + +This module is Layer 2 and imports the standard library only. Nothing under +``molmcp.host`` may import ``molmcp.client_config``, ``molmcp.cli``, +``molmcp.server``, ``molmcp.providers``, or ``molmcp.discovery``: +``client_config`` reads this table, not the other way round. +""" + +from __future__ import annotations + +import re +from collections.abc import Mapping +from dataclasses import dataclass +from pathlib import Path +from types import MappingProxyType +from typing import Literal + +Host = Literal["grok", "claude", "cursor", "codex"] +"""The hosts ``molmcp init`` knows how to wire.""" + +SKILL_NAME = "molcrafts" +"""Directory name of the managed usage skill inside a host's ``skills/``.""" + + +@dataclass(frozen=True, slots=True) +class HostLayout: + """Where one host keeps every file ``molmcp init`` may write. + + Each field is a path tuple relative to ``Path.home()``, which keeps a + record immutable, hashable, and independent of the home directory in + force when it is read. + + Attributes: + mcp_json: The host's MCP (Model Context Protocol) client config — + the list of servers it launches, and the only JSON + ``molmcp init`` writes. + skill_dir: Directory holding the managed usage constitution + ``SKILL.md``. Its last part is always :data:`SKILL_NAME`. + adapter: Stable pointer file ``molmcp-adapter.md``. + agents: Host agents root. Only + :func:`~molmcp.host.place.place_components` writes there, and + only the ``agent`` components a catalog declares by name, so a + user's own files are left alone. + rules: Host rules root. Written on the same terms as *agents*, for + ``rule`` components. + """ + + mcp_json: tuple[str, ...] + skill_dir: tuple[str, ...] + adapter: tuple[str, ...] + agents: tuple[str, ...] + rules: tuple[str, ...] + + +HOSTS: dict[Host, HostLayout] = { + "grok": HostLayout( + mcp_json=(".mcp.json",), + skill_dir=(".grok", "skills", SKILL_NAME), + adapter=(".grok", "molmcp-adapter.md"), + agents=(".grok", "agents"), + rules=(".grok", "rules"), + ), + "claude": HostLayout( + mcp_json=(".claude.json",), + skill_dir=(".claude", "skills", SKILL_NAME), + adapter=(".claude", "molmcp-adapter.md"), + agents=(".claude", "agents"), + rules=(".claude", "rules"), + ), + "cursor": HostLayout( + mcp_json=(".cursor", "mcp.json"), + skill_dir=(".cursor", "skills", SKILL_NAME), + adapter=(".cursor", "molmcp-adapter.md"), + agents=(".cursor", "agents"), + rules=(".cursor", "rules"), + ), + "codex": HostLayout( + mcp_json=(".codex", "mcp.json"), + skill_dir=(".codex", "skills", SKILL_NAME), + adapter=(".codex", "molmcp-adapter.md"), + agents=(".codex", "agents"), + rules=(".codex", "rules"), + ), +} +"""Layout per host, in the order ``molmcp init`` offers as ``--help`` choices.""" + + +def layout_for(host: Host) -> HostLayout: + """Return the layout record for *host*. + + Args: + host: One of the keys of :data:`HOSTS`. + + Returns: + The immutable :class:`HostLayout` describing that host's paths. + + Raises: + ValueError: If *host* is not a known host. + """ + if host not in HOSTS: + known = ", ".join(sorted(HOSTS)) + raise ValueError(f"unknown host {host!r}; known: {known}") + return HOSTS[host] + + +def default_write_path(host: Host) -> Path: + """Conventional destination for *host*'s MCP config. + + Args: + host: One of the keys of :data:`HOSTS`. + + Returns: + ``Path.home()`` joined with the record's ``mcp_json`` parts. + + Raises: + ValueError: If *host* is not a known host. + """ + return Path.home().joinpath(*layout_for(host).mcp_json) + + +def default_skill_dir(host: Host) -> Path: + """User-level skill directory for *host* (``SKILL.md`` lives inside). + + Args: + host: One of the keys of :data:`HOSTS`. + + Returns: + ``Path.home()`` joined with the record's ``skill_dir`` parts. + + Raises: + ValueError: If *host* is not a known host. + """ + return Path.home().joinpath(*layout_for(host).skill_dir) + + +_KEY_LINE = re.compile(r"^([A-Za-z0-9][A-Za-z0-9_-]*):(.*)$") + +_FRONTMATTER_MAPS: Mapping[Host, Mapping[str, str]] = MappingProxyType( + { + "grok": MappingProxyType( + { + "name": "name", + "description": "description", + "when-to-use": "when-to-use", + "user-invocable": "user-invocable", + "disable-model-invocation": "disable-model-invocation", + "argument-hint": "argument-hint", + } + ), + "claude": MappingProxyType( + { + "name": "name", + "description": "description", + "user-invocable": "user-invocable", + "disable-model-invocation": "disable-model-invocation", + "argument-hint": "argument-hint", + } + ), + "cursor": MappingProxyType( + { + "name": "name", + "description": "description", + "disable-model-invocation": "disable-model-invocation", + } + ), + "codex": MappingProxyType( + { + "name": "name", + "description": "description", + } + ), + } +) + + +def remap_frontmatter(text: str, host: Host) -> str: + """Rewrite top-level YAML keys for *host*; drop unmapped keys. + + Line-oriented: values are not parsed. A document without a closed + ``---`` fence is returned unchanged. Output uses ``\\n`` newlines. + + Args: + text: File contents, typically a SKILL.md / agent / rule body. + host: Destination host; validated via :func:`layout_for`. + + Returns: + Remapped text, or *text* when there is no closed fence. + + Raises: + ValueError: If *host* is not a known host. + """ + layout_for(host) + mapping = _FRONTMATTER_MAPS[host] + lines = text.splitlines() + if not lines or lines[0].rstrip() != "---": + return text + close: int | None = None + for index, line in enumerate(lines[1:], start=1): + if line.rstrip() == "---": + close = index + break + if close is None: + return text + blocks: list[tuple[str, list[str]]] = [] + current_key: str | None = None + current_lines: list[str] = [] + for line in lines[1:close]: + match = None + if not line.startswith((" ", "\t")): + match = _KEY_LINE.match(line) + if match is not None: + if current_key is not None: + blocks.append((current_key, current_lines)) + current_key = match.group(1) + current_lines = [line] + continue + if current_key is None: + continue + current_lines.append(line) + if current_key is not None: + blocks.append((current_key, current_lines)) + out = ["---"] + for key, block in blocks: + dest = mapping.get(key) + if dest is None: + continue + first = block[0] + rest = first.split(":", 1)[1] + out.append(f"{dest}:{rest}") + out.extend(block[1:]) + out.append("---") + out.extend(lines[close + 1 :]) + return "\n".join(out) + "\n" diff --git a/src/molmcp/host/place.py b/src/molmcp/host/place.py new file mode 100644 index 0000000..0eb3b28 --- /dev/null +++ b/src/molmcp/host/place.py @@ -0,0 +1,288 @@ +"""Placing catalog-declared component files into a host's directories. + +``molmcp harness sync`` publishes a commit tree and moves a source's +activation pointer onto it, and ``molmcp init`` then has to install whatever +that tree's catalog declares. This module is the last link of that chain and +owns exactly one fact: which host directory a component *kind* belongs in. + +Nothing here learns what a catalog is. The caller owns catalog grammar — it +strips each kind's path prefix, joins the component root of the source a row +came from, and hands over one :class:`ComponentFile` per file to place: four +plain values, no catalog type among them. ``host/`` answers with the one +thing the caller cannot know, the kind table below, which is why +:class:`ComponentFile` validates no kind at construction and +:func:`place_components` refuses an unknown one. + +Two rules are load-bearing: + +* **The tree is never globbed.** :func:`place_components` copies the files it + is handed and reads no other path, so a file sitting beside a declared + component that no catalog row mentions is not a component and cannot reach + a host. +* **The managed usage skill is never clobbered.** + :func:`~molmcp.host.install.install_skill` owns the constitution under + :data:`~molmcp.host.layout.SKILL_NAME`; a row aimed there is skipped, which + is the protection :func:`~molmcp.host.install.materialize_daily` already + applies on the checkout route. + +This module is Layer 2 and imports the standard library only. Nothing under +``molmcp.host`` may import ``molmcp.components``, ``molmcp.harness``, +``molmcp.client_config``, ``molmcp.cli``, ``molmcp.server``, +``molmcp.providers``, or ``molmcp.discovery``: a component arrives here as +four stdlib values precisely so none of them is needed. +""" + +from __future__ import annotations + +from collections.abc import Callable, Mapping, Sequence +from dataclasses import dataclass +from pathlib import Path, PurePosixPath +from types import MappingProxyType + +from .layout import Host, HostLayout, layout_for, remap_frontmatter + +SKIP_NO_HOST_DESTINATION = "kind has no host destination" +"""Why a ``provider`` or ``overlay`` row is reported but not installed. + +A provider is a plane ``molmcp serve`` mounts and an overlay is knowledge the +discovery index reads; neither is a file any host keeps, so both are declared +by a catalog and refused by this seam. +""" + +SKIP_MANAGED_USAGE_SKILL = "managed usage skill is owned by molmcp init" +"""Why a row landing inside the managed usage skill is refused. + +:func:`~molmcp.host.install.install_skill` writes that constitution, so a +catalog cannot take the name from it however it spells the path. +""" + + +@dataclass(frozen=True, slots=True) +class ComponentFile: + """One file a catalog declared, already resolved by the caller. + + Frozen, so a description cannot be edited between the pre-flight pass and + the copy that trusts it. + + Attributes: + id: The catalog id, unchanged — ``"skill.daily"``. Used for reports + and error messages only; nothing is placed by it. + kind: The catalog kind as a plain string. It is deliberately not + validated here: which kinds have a host destination is + :func:`place_components`' table, and that table has one owner. + relative: POSIX path of the file inside its kind's host directory, + with the catalog's kind prefix already stripped — + ``"daily/SKILL.md"``, not ``"skills/daily/SKILL.md"``. + source: Absolute path of the file to copy, inside the activated + commit tree. The caller has already joined the component root of + the source this row came from, so a fold spanning several sources + resolves every row under its own base. + """ + + id: str + kind: str + relative: str + source: Path + + +@dataclass(frozen=True, slots=True) +class PlacementReport: + """What one :func:`place_components` run placed, replaced, and refused. + + A bare tuple of paths would hide the two decisions a run makes, so both + are recorded: that a destination already existed, and that a component + was skipped rather than installed. + + Attributes: + installed: Destinations written, in the order the components were + given. + replaced: The subset of *installed* that already existed before the + run, in the same order. Empty on a first run; equal to + *installed* on a repeat of the same set. + skipped: One ``(component id, reason)`` pair per refusal, in input + order. The reason is :data:`SKIP_NO_HOST_DESTINATION` or + :data:`SKIP_MANAGED_USAGE_SKILL`. + """ + + installed: tuple[Path, ...] + replaced: tuple[Path, ...] + skipped: tuple[tuple[str, str], ...] + + +_KIND_ROOTS: Mapping[str, Callable[[HostLayout], tuple[str, ...]]] = MappingProxyType( + { + # ``skill_dir`` names the managed usage skill itself, so its parent is + # the host's ``skills/`` — read off the one layout table rather than + # spelled again here. + "skill": lambda layout: layout.skill_dir[:-1], + "agent": lambda layout: layout.agents, + "rule": lambda layout: layout.rules, + } +) +"""Which host directory each installable kind belongs in. + +The single owner of that question. A caller resolving component paths knows +the catalog's own layout and nothing about a host's, which is why this table +lives on this side of the seam. +""" + +_KINDS_WITHOUT_HOST_DESTINATION: frozenset[str] = frozenset({"provider", "overlay"}) +"""Declared kinds that no host directory holds.""" + + +def _host_root(component: ComponentFile, layout: HostLayout) -> tuple[str, ...] | None: + """Layout path parts of the directory that holds *component*'s kind. + + Args: + component: The description whose ``kind`` is being placed. + layout: The target host's layout record. + + Returns: + The path tuple of that kind's host directory, relative to home, or + ``None`` when the kind has no host destination at all. + + Raises: + ValueError: If the kind is not one of the five catalog kinds. A sixth + means the caller is broken, not the file. + """ + resolve = _KIND_ROOTS.get(component.kind) + if resolve is not None: + return resolve(layout) + if component.kind in _KINDS_WITHOUT_HOST_DESTINATION: + return None + known = ", ".join(sorted({*_KIND_ROOTS, *_KINDS_WITHOUT_HOST_DESTINATION})) + raise ValueError( + f"component {component.id!r} has unknown kind {component.kind!r}; " + f"known: {known}" + ) + + +def _destination(root: Path, component: ComponentFile) -> Path: + """Resolve *component*'s ``relative`` under *root*, refusing any escape. + + The path is read as POSIX because that is the grammar a catalog is + written in, and the result is required to sit strictly inside *root*, so + neither ``..`` nor an absolute path can steer a write out of the host. + + Args: + root: Absolute host directory for the component's kind. + component: The description being placed. + + Returns: + The absolute destination path, uncreated. + + Raises: + ValueError: If ``relative`` is empty, absolute, or leaves *root*. + """ + relative = PurePosixPath(component.relative) + destination = root.joinpath(*relative.parts) + escapes = ( + relative.is_absolute() + or ".." in relative.parts + or root not in destination.parents + ) + if escapes: + raise ValueError( + f"component {component.id!r} places {component.relative!r} outside {root}" + ) + return destination + + +def _require_file(component: ComponentFile) -> None: + """Fail unless *component*'s source is an existing regular file. + + Args: + component: The description being placed. + + Raises: + FileNotFoundError: If the source is missing or is not a file, naming + both the component id and the path so the broken catalog row is + identifiable from the message alone. + """ + if not component.source.is_file(): + raise FileNotFoundError( + f"component {component.id!r} is not a file: {component.source}" + ) + + +def place_components( + host: Host, components: Sequence[ComponentFile] +) -> PlacementReport: + """Install every component that has a host destination into *host*. + + Each component is copied to its kind's host directory, in the order + given. Only the files handed over are read: no directory is walked, so + what reaches the host is exactly what a catalog declared. Every source is + checked in a pre-flight pass before the first byte is written, so one + unresolvable row leaves no partial set behind. Re-running with the same + components rewrites the same destinations and reports them as + ``replaced``. + + Args: + host: One of the known hosts. Validated first, as in every other + primitive of this family, so a broken host name raises even when + there is nothing to place. + components: Descriptions of the files to place, already resolved + against the activated commit tree by the caller. + + Returns: + A :class:`PlacementReport` naming what was written, what it replaced, + and which components were refused with which reason. + + Raises: + ValueError: If *host* is unknown, if a component's kind is unknown, + or if a component's ``relative`` path escapes its host directory. + FileNotFoundError: If a component's source is missing or is not a + file; raised before anything is written. + OSError: If a destination cannot be written. + """ + layout = layout_for(host) + home = Path.home() + managed = home.joinpath(*layout.skill_dir) + + planned: list[tuple[ComponentFile, Path]] = [] + skipped: list[tuple[str, str]] = [] + for component in components: + parts = _host_root(component, layout) + if parts is None: + skipped.append((component.id, SKIP_NO_HOST_DESTINATION)) + continue + destination = _destination(home.joinpath(*parts), component) + if destination == managed or managed in destination.parents: + skipped.append((component.id, SKIP_MANAGED_USAGE_SKILL)) + continue + planned.append((component, destination)) + + for component, _ in planned: + _require_file(component) + rewritten: list[tuple[ComponentFile, Path, str]] = [] + for component, destination in planned: + rewritten.append( + (component, destination, component.source.read_text(encoding="utf-8")) + ) + + installed: list[Path] = [] + replaced: list[Path] = [] + for component, destination, text in rewritten: + if destination.exists(): + replaced.append(destination) + destination.parent.mkdir(parents=True, exist_ok=True) + destination.write_text( + remap_frontmatter(text, host), encoding="utf-8", newline="\n" + ) + installed.append(destination) + + return PlacementReport( + installed=tuple(installed), + replaced=tuple(replaced), + skipped=tuple(skipped), + ) + + +__all__ = [ + "SKIP_MANAGED_USAGE_SKILL", + "SKIP_NO_HOST_DESTINATION", + "ComponentFile", + "PlacementReport", + "place_components", +] diff --git a/src/molmcp/mcp_provider.py b/src/molmcp/mcp_provider.py index 617c9b3..c5063d7 100644 --- a/src/molmcp/mcp_provider.py +++ b/src/molmcp/mcp_provider.py @@ -39,11 +39,12 @@ class MolCraftsContextProvider: - """Register hierarchical discovery tools on the **molcrafts** plane. + """Register hierarchical discovery tools on the **molcrafts** core. Tool names are bare (``packages``, ``open``, …). The MCP server name is ``molcrafts``, so clients see ``molcrafts__packages`` — never a mega - ``molmcp__molcrafts_*`` prefix stack. + ``molmcp__molcrafts_*`` prefix stack. ``list_planes`` / ``route`` are + registered alongside these tools by ``create_plane``. """ name = "molcrafts" diff --git a/src/molmcp/planes.py b/src/molmcp/planes.py index ae75db9..959df47 100644 --- a/src/molmcp/planes.py +++ b/src/molmcp/planes.py @@ -1,21 +1,12 @@ """MCP planes — one product domain per MCP server process. -Each plane is an independent MCP server identity. Client configs default to -**all planes enabled**; operators toggle with ``--enable`` / ``--disable``. -There is no mega-server that mounts every provider under one ``molmcp`` name. - -Planes ------- -catalog - Bootstrap only. Lists available planes and routes a task string to which - plane(s) to connect. No science, no discovery index. -molcrafts - Knowledge plane: packages / outline / open / search / compose / suggest. - Science APIs are discovered here and invoked elsewhere (e.g. molvis exec). -molvis / molq / molexp / … - Stateful provider planes from ``molmcp.providers`` entry points. Each - process hosts exactly one provider's tools, with bare tool names - (client sees ``molvis__open``, not ``molmcp__molvis_open``). +``molcrafts`` is the **core** connection: knowledge pages plus +``list_planes`` / ``route``. It is always on and cannot be disabled. +Provider planes (``molvis`` / ``molq`` / ``molexp`` / …) are optional +MCP links from the ``molmcp.providers`` entry-point group. + +There is no catalog plane. Default ``molmcp serve`` is the molcrafts core +with enabled providers FastMCP-mounted (namespaced tools). """ from __future__ import annotations @@ -23,13 +14,20 @@ from dataclasses import dataclass from typing import Any -from .provider import discover_providers +from .provider import Provider, discover_providers + +#: Always-on knowledge + routing connection. Not a disableable plane. +CORE_PLANE_ID = "molcrafts" + +#: Built-in ids that are not entry-point providers. +BUILTIN_PLANE_IDS = frozenset({CORE_PLANE_ID}) -#: Built-in planes that are not entry-point providers. -BUILTIN_PLANE_IDS = frozenset({"catalog", "molcrafts"}) +#: Retired plane id. Kept out of catalogs; serving it fails loudly. +GONE_PLANE_IDS = frozenset({"catalog"}) -#: Intent routing table for the catalog ``route`` tool. +#: Intent routing table for the ``route`` tool. #: Patterns are lowercase substrings matched against the task string. +#: Only **provider** planes appear here — the core is already connected. _ROUTE_HINTS: tuple[tuple[tuple[str, ...], str, str], ...] = ( ( ( @@ -87,37 +85,36 @@ "molexp", "Experiment workspace layout, scaffold, and legacy-directory adoption.", ), - ( - ( - "api", - "symbol", - "docstring", - "import", - "how to", - "search code", - "package", - "查", - "文档", - "符号", - "接口", - ), - "molcrafts", - "Discover package/module/symbol pages before writing code.", - ), ) +def gone_plane_message(plane_id: str) -> str: + """Loud error when a retired plane id is used.""" + if plane_id == "catalog": + return ( + "catalog is not a plane; list_planes and route live on molcrafts. " + "Use `molmcp serve molcrafts`." + ) + return f"{plane_id!r} is not a plane" + + +def core_disable_message() -> str: + """Loud error when the caller tries to disable the core connection.""" + return "molcrafts is the core connection and cannot be disabled" + + @dataclass(frozen=True, slots=True) class PlaneInfo: - """Public description of one connectable MCP plane.""" + """Public description of one connectable MCP server.""" id: str - kind: str # "builtin" | "provider" + kind: str # "core" | "provider" purpose: str when_to_connect: str serve_command: str requires_config: bool tools_hint: tuple[str, ...] + disableable: bool def to_dict(self) -> dict[str, Any]: return { @@ -128,35 +125,27 @@ def to_dict(self) -> dict[str, Any]: "serve_command": self.serve_command, "requires_config": self.requires_config, "tools_hint": list(self.tools_hint), + "disableable": self.disableable, } -def _catalog_info() -> PlaneInfo: - return PlaneInfo( - id="catalog", - kind="builtin", - purpose="List planes and route a task to which MCP connection(s) to open.", - when_to_connect=( - "Bootstrap routing; safe to leave enabled with everything else." - ), - serve_command="molmcp serve catalog", - requires_config=False, - tools_hint=("list_planes", "route"), - ) - - def _molcrafts_info() -> PlaneInfo: return PlaneInfo( - id="molcrafts", - kind="builtin", - purpose="Inject knowledge pages (packages → outline → open → compose).", + id=CORE_PLANE_ID, + kind="core", + purpose=( + "Always-on knowledge pages (packages → outline → open → compose) " + "plus list_planes / route for optional provider planes." + ), when_to_connect=( - "Before writing science code: discover real symbols and examples. " - "Never invent APIs; miss means SYMBOL_NOT_FOUND." + "Core connection — always on. Discover real symbols before writing " + "code. Never invent APIs; miss means SYMBOL_NOT_FOUND." ), - serve_command="molmcp serve molcrafts", + serve_command="molmcp serve", requires_config=True, tools_hint=( + "list_planes", + "route", "info", "packages", "outline", @@ -165,89 +154,102 @@ def _molcrafts_info() -> PlaneInfo: "search", "suggest", ), + disableable=False, ) -_PROVIDER_META: dict[str, tuple[str, str, tuple[str, ...]]] = { +#: Product copy for the planes molmcp itself ships, keyed by plane id. +#: **Not a membership table** — a key here is only ever looked up for a name +#: :func:`discover_providers` already reported. Holding a row for a plane the +#: entry-point group never registered must not put it in a catalog, because +#: ``molmcp serve`` could not start it. +_PROVIDER_COPY: dict[str, tuple[str, str]] = { "molvis": ( "Live molvis viewer: persistent Python namespace + browser canvas.", "User wants to draw, load, select, or interact with a molecule in 3D.", - ( - "open", - "exec", - "poll_events", - "list_sessions", - "capabilities", - "refresh", - "close", - ), ), "molq": ( "molq job lifecycle: list/get/logs destinations; opt-in submit/cancel.", "User wants cluster jobs, queue status, or submission.", - ("list_jobs", "get_job", "job_logs", "list_destinations", "list_queue"), ), "molexp": ( "molexp workspace navigation, idempotent scaffold, and adoption of a " "legacy data directory (not a run driver).", "User works with experiment workspaces, projects, FAIR layout, or has " "a folder of results to lift into one.", - ( - "list_projects", - "list_experiments", - "list_runs", - "workspace_layout", - "validate_workspace", - "materialize_workspace", - "add_project", - "add_experiment", - "create_run", - "validate_workflow", - "plan_adoption", - "run_adoption", - "adoption_status", - "ingest_metrics", - ), ), } +def _provider_copy(name: str) -> tuple[str, str]: + """Return the ``(purpose, when_to_connect)`` sentences for a member plane. + + Args: + name: Plane id, as the entry-point group reported it. + + Returns: + The catalog's own product copy when ``name`` has a row, otherwise a + generic pair naming the plane and the group it came from. + """ + return _PROVIDER_COPY.get( + name, + ( + f"Provider plane '{name}' (entry point molmcp.providers).", + f"When work needs the '{name}' product surface.", + ), + ) + + +def _tools_hint(provider: Provider) -> tuple[str, ...]: + """Return the tool names *provider* publishes about itself. + + Duck-typed exactly like ``probe`` is in ``provider_available``: the + instance already answers this, so the catalog keeps no parallel tool + list that could drift away from what ``register`` actually attaches. + + Args: + provider: A discovered provider instance. + + Returns: + The wire names from ``provider.tool_specs()``, or an empty tuple + when the instance does not publish specs (the Protocol minimum). + """ + specs_fn = getattr(provider, "tool_specs", None) + if not callable(specs_fn): + return () + return tuple(spec.name for spec in specs_fn()) + + def list_plane_infos(*, include_unavailable_providers: bool = False) -> list[PlaneInfo]: - """Return planes this install can serve (catalog first). + """Return the core connection plus the provider planes the group reports. - By default only providers whose optional upstream package is installed - appear (**silent omit** of missing science deps — not a test skip). - Pass ``include_unavailable_providers=True`` for diagnostics. + Membership has exactly one authority: the ``molmcp.providers`` + entry-point group, read through :func:`discover_providers`. By default + only providers whose optional upstream package is installed appear + (**silent omit** of missing science deps — not a test skip). + + Args: + include_unavailable_providers: Widen discovery to providers whose + ``probe()`` is false, for diagnostics. It widens availability + only — a name the group never registered is still never listed. + + Returns: + The core plane first, then one row per discovered provider, by id. """ - planes: list[PlaneInfo] = [_catalog_info(), _molcrafts_info()] - available = {p.name: p for p in discover_providers(only_available=True)} - if include_unavailable_providers: - loaded = {p.name: p for p in discover_providers(only_available=False)} - names = sorted(set(loaded) | set(_PROVIDER_META)) - by_name = loaded - else: - names = sorted(available) - by_name = available - for name in names: - if name not in by_name and not include_unavailable_providers: - continue - purpose, when, tools = _PROVIDER_META.get( - name, - ( - f"Provider plane '{name}' (entry point molmcp.providers).", - f"When work needs the '{name}' product surface.", - (), - ), - ) + planes: list[PlaneInfo] = [_molcrafts_info()] + discovered = discover_providers(only_available=not include_unavailable_providers) + for provider in sorted(discovered, key=lambda member: member.name): + purpose, when = _provider_copy(provider.name) planes.append( PlaneInfo( - id=name, + id=provider.name, kind="provider", purpose=purpose, when_to_connect=when, - serve_command=f"molmcp serve {name}", + serve_command=f"molmcp serve {provider.name}", requires_config=False, - tools_hint=tools, + tools_hint=_tools_hint(provider), + disableable=True, ) ) return planes @@ -261,14 +263,13 @@ def known_plane_ids(*, only_available: bool = False) -> frozenset[str]: ``register``). Catalogs use *only_available*. """ provider_names = {p.name for p in discover_providers(only_available=only_available)} - if only_available: - return frozenset(BUILTIN_PLANE_IDS | provider_names) - return frozenset(BUILTIN_PLANE_IDS | provider_names | set(_PROVIDER_META)) + return frozenset(BUILTIN_PLANE_IDS | provider_names) def route_task(task: str) -> dict[str, Any]: - """Map a free-text task to plane ids the client should connect. + """Map a free-text task to optional provider planes to connect. + ``molcrafts`` is the core and is never returned as a plane to add. Returns a structured routing answer — never executes science. """ text = task.strip().lower() @@ -278,40 +279,30 @@ def route_task(task: str) -> dict[str, Any]: if any(k in text for k in keywords) and plane_id not in seen: seen.add(plane_id) matched.append({"plane": plane_id, "reason": reason}) - # Default: knowledge first when nothing matched hard. - if not matched: - matched.append( - { - "plane": "molcrafts", - "reason": "No strong product signal; discover APIs before coding.", - } - ) - # Drawing almost always needs molcrafts for API truth + molvis for canvas. - plane_ids = [m["plane"] for m in matched] - if "molvis" in plane_ids and "molcrafts" not in plane_ids: - matched.append( - { - "plane": "molcrafts", - "reason": "Look up molpy/molvis symbols before writing exec code.", - } - ) return { "ok": True, "task": task, + "core": CORE_PLANE_ID, "planes": matched, - "serve_commands": [f"molmcp serve {m['plane']}" for m in matched], + "namespaces": [m["plane"] for m in matched], + "serve_commands": ["molmcp serve"], "client_hint": ( - "Default client installs every plane; use " - "`molmcp client grok --disable …` to drop ones you do not want. " - "Science APIs are never MCP tools — discover them on the " - "molcrafts plane, then call them inside molvis exec (or agent Python)." + "Default `molmcp serve` already mounts these providers onto " + "molcrafts (molvis_open, molq_list_jobs, …). Omit a mount with " + "`molmcp init grok --disable …`. Science APIs are never MCP " + "tools — discover them on molcrafts, then call them in agent " + "Python or molvis_exec." ), } __all__ = [ "BUILTIN_PLANE_IDS", + "CORE_PLANE_ID", + "GONE_PLANE_IDS", "PlaneInfo", + "core_disable_message", + "gone_plane_message", "known_plane_ids", "list_plane_infos", "route_task", diff --git a/src/molmcp/provider.py b/src/molmcp/provider.py index de5b0f7..2224713 100644 --- a/src/molmcp/provider.py +++ b/src/molmcp/provider.py @@ -5,12 +5,15 @@ import importlib.metadata import logging import re -from typing import Protocol, runtime_checkable +from typing import TYPE_CHECKING, Protocol, runtime_checkable -from fastmcp import FastMCP +if TYPE_CHECKING: + from fastmcp import FastMCP PROVIDER_ENTRY_POINT_GROUP = "molmcp.providers" PROVIDER_NAME_PATTERN = re.compile(r"^[a-z][a-z0-9-]*$") +# ``catalog`` is retired as a plane; keep the name reserved so no provider +# can occupy the old server id. RESERVED_PROVIDER_NAMES = frozenset({"molcrafts", "catalog"}) logger = logging.getLogger(__name__) @@ -22,12 +25,12 @@ class Provider(Protocol): Implementations must expose: - * ``name`` — plane id and MCP server name (e.g. ``"molvis"``). Clients - connect with ``molmcp serve ``; tool ids become - ``__`` at the client (e.g. ``molvis__open``). - * ``register(mcp)`` — attach **bare** tool names only (``open``, not - ``molvis_open``). Never mount with a namespace. Startup rejects - ``{plane}_…`` and ``{plane}_{plane}_…`` (legacy ``molexp_molexp_*``). + * ``name`` — plane id (e.g. ``"molvis"``). On the composed core, FastMCP + namespaces tools as ``molvis_open``. A debug ``molmcp serve molvis`` + process still uses bare ``open`` (client ``molvis__open``). + * ``register(mcp)`` — attach **bare** tool names only (``open``). The + parent ``create_stack`` adds the namespace. A focused process named + ``molvis`` still rejects registering ``molvis_open`` (would double). Optional: diff --git a/src/molmcp/provider_sdk.py b/src/molmcp/provider_sdk.py new file mode 100644 index 0000000..45efc17 --- /dev/null +++ b/src/molmcp/provider_sdk.py @@ -0,0 +1,336 @@ +"""Public Provider SDK — declare tools, probe upstream, register on FastMCP. + +MCP (Model Context Protocol) is the wire protocol an AI client uses to call +tools. FastMCP is the Python library that hosts an MCP server. A *plane* is +one product's MCP server (``molvis``, ``molq``, ``molexp``). A *Provider* is +the class that declares that plane's tools. + +A client decides whether to call a tool unattended from MCP +``ToolAnnotations``: read-only vs writing, local vs open-world (the call +reaches a network, scheduler, or browser), destructive vs additive, +idempotent vs not. Six named constants in this module cover every +first-party tool. + +*probe* is an import-system availability check: ``importlib.util.find_spec`` +asks whether an upstream science package *could* be imported, without +importing it. Catalogs omit a missing plane; only an explicit +``molmcp serve `` fails loudly. + +Providers are discovered through the ``molmcp.providers`` entry-point group +(a packaging hook that lists Provider classes). The entry-point name must +equal :attr:`ProviderBase.name`. Tools always register *bare* (``open``, +never ``molvis_open``). Clients then see two forms: a focused +``molmcp serve molvis`` / :func:`~molmcp.create_plane` process shows +``molvis__open``; the composed ``molmcp serve`` / :func:`~molmcp.create_stack` +core mounts with a FastMCP namespace, so clients show ``molvis_open`` (some +clients ``molcrafts__molvis_open``). + +A plane author subclasses :class:`ProviderBase`, marks methods with +:func:`tool`, and picks annotations from this module. The runtime protocol +:class:`Provider` is defined in :mod:`molmcp.provider` and re-exported here; +entry-point discovery, namespace authority, and availability filtering stay +there. + +Each provider used to spend most of its class on one ``register()`` method — +349, 402 and 191 lines — holding every tool as a nested function, plus its +own copy of the availability probe, the missing-package guard, and a set of +hand-rolled annotations. The duplication drifted: three probes with three +signatures, three guard messages (one of which never said how to install +anything), and annotation values that disagreed between planes. + +Here a tool is a method carrying a :func:`tool` declaration. The base +collects them, checks the upstream package once, and registers. Providers +are left holding only what is theirs: what the tools do. + +Nothing here imports a science package. Importing it to find out whether it +exists would drag a whole scientific stack into a process that only wanted +to print a list. + +A provider that needs a seventh annotation should add it here, with the +reason, rather than build one inline — the whole point is that a named +vocabulary cannot drift the way inline literals did. +""" + +from __future__ import annotations + +import importlib.util +from collections.abc import Callable, Iterator +from dataclasses import dataclass +from typing import TYPE_CHECKING, Any, ClassVar + +from mcp.types import ToolAnnotations + +from .provider import Provider + +if TYPE_CHECKING: + from fastmcp import FastMCP + +#: Attribute a declared tool carries. Private by convention; read only here. +_MARKER = "__molmcp_tool__" + +#: Reads local state and nothing else. Safe to call, safe to repeat. +READ_ONLY = ToolAnnotations( + read_only_hint=True, + destructive_hint=False, + idempotent_hint=True, + open_world_hint=False, +) + +#: Reads, but reaches a scheduler, a browser, or the network to do it +#: (open-world: ``open_world_hint=True``). Still safe to call; the answer +#: can change underneath you. +READ_REMOTE = ToolAnnotations( + read_only_hint=True, + destructive_hint=False, + idempotent_hint=False, + open_world_hint=True, +) + +#: Changes state beyond this machine and cannot be trivially undone — +#: submitting to a cluster, cancelling a remote job, driving a browser. A +#: client should confirm before calling one of these. +MUTATION = ToolAnnotations( + read_only_hint=False, + destructive_hint=True, + idempotent_hint=False, + open_world_hint=True, +) + +#: Rewrites or removes local state, and resumes rather than duplicating when +#: called again. +#: +#: Destructiveness and reach are independent axes, and the first cut of this +#: vocabulary fused them: every destructive tool had to claim it touched an +#: open world (``open_world_hint=True``: the call reaches a network, +#: scheduler, or browser). molexp's ``run_adoption`` is the case that +#: exposed it — move mode unlinks source files, it resumes from a ledger, +#: and it never leaves the filesystem. Forcing it onto MUTATION would have +#: made it lie twice. +LOCAL_MUTATION = ToolAnnotations( + read_only_hint=False, + destructive_hint=True, + idempotent_hint=True, + open_world_hint=False, +) + +#: Adds to a local record; calling it twice adds twice. +#: +#: Additive, so *not* destructive — MCP ``ToolAnnotations`` treats +#: ``destructive_hint`` and a purely additive write as opposites. What a +#: caller needs to know is that a retry is not free, which is what +#: ``idempotent_hint=False`` says. Flagging it destructive instead would make +#: a client confirm every append, which is noise. +APPEND_WRITE = ToolAnnotations( + read_only_hint=False, + destructive_hint=False, + idempotent_hint=False, + open_world_hint=False, +) + +#: Create-or-get. Writes, but calling it twice leaves the same state, so it +#: is not a destructive surface even though it is not a read. +IDEMPOTENT_WRITE = ToolAnnotations( + read_only_hint=False, + destructive_hint=False, + idempotent_hint=True, + open_world_hint=False, +) + + +@dataclass(frozen=True, slots=True) +class ToolSpec: + """One tool a provider offers. + + Attributes: + name: Bare tool identifier registered on the server (``open``, never + ``molvis_open``). A focused process is named after the plane, so + clients show ``__``; the composed core mounts with a + FastMCP namespace, so clients show ``_``. + annotations: MCP ``ToolAnnotations`` the client uses to decide + whether to confirm before calling. Use one of :data:`READ_ONLY`, + :data:`READ_REMOTE`, :data:`MUTATION`, :data:`LOCAL_MUTATION`, + :data:`APPEND_WRITE`, :data:`IDEMPOTENT_WRITE`. + attribute: Name of the method implementing it. + """ + + name: str + annotations: ToolAnnotations + attribute: str + + +def tool( + annotations: ToolAnnotations, *, name: str | None = None +) -> Callable[[Callable[..., Any]], Callable[..., Any]]: + """Declare a method as one of this plane's MCP tools. + + Args: + annotations: What a client needs to decide whether to confirm first. + Use a constant from this module. + name: Bare wire identifier (``open``, not ``molvis_open``). Required + when the method cannot be that identifier (``open`` shadows a + builtin, ``exec`` is a keyword); otherwise defaults to the + method name. Startup rejects a prefixed name. + + Returns: + Decorator that records ``(name, annotations)`` on the method and + returns that method unchanged (not a wrapper), so FastMCP sees the + original signature and docstring. + """ + + def declare(fn: Callable[..., Any]) -> Callable[..., Any]: + setattr(fn, _MARKER, (name or fn.__name__, annotations)) + return fn + + return declare + + +class ProviderBase: + """Base for a plane's provider. + + Subclasses set :attr:`name`, optionally :attr:`upstream` / + :attr:`import_name`, and declare tools with :func:`tool`. + + Attributes: + name: Plane id and MCP server name. Must equal the + ``molmcp.providers`` entry-point name. Tools still register + bare. Clients then see two forms: ``__`` on a + focused ``molmcp serve `` process, and ``_`` + (FastMCP namespace) on the composed core (``create_stack`` / + ``molmcp serve``). + upstream: Distribution to install when the plane is unavailable, as + it would be typed after ``pip install``. ``None`` means the plane + needs nothing beyond molmcp. + import_name: Module name :meth:`probe` passes to + ``importlib.util.find_spec``. When omitted, hyphens in + *upstream* become underscores (``molcrafts-foo`` → + ``molcrafts_foo``). Set this when the importable module is a + different name (``upstream='molcrafts-molq'``, + ``import_name='molq'``). + """ + + name: ClassVar[str] + upstream: ClassVar[str | None] = None + import_name: ClassVar[str | None] = None + + # -- availability ------------------------------------------------- + + def probe(self) -> bool: + """Whether this plane can be served here. + + A plane whose science package is missing is a normal state, not an + error: plane catalogs and generated client configs omit it silently. + Only an explicit ``molmcp serve `` fails, and then loudly. + + Override when availability is not just "the package is present" — + molvis is available whenever a caller-supplied stage factory (the + function that builds the viewer, used by embedders and tests) has + been injected, browser or no browser. + + Returns: + True if this plane can be served here: no ``upstream``, or + ``importlib.util.find_spec`` finds ``import_name`` (or + ``upstream`` with hyphens turned into underscores). A missing + or broken package yields False — catalogs omit the plane; only + an explicit ``molmcp serve `` fails loudly. + """ + module = self.import_name or ( + self.upstream.replace("-", "_") if self.upstream else None + ) + if module is None: + return True + try: + return importlib.util.find_spec(module) is not None + except (ImportError, ValueError): + # A package present but broken is not one we can serve. + return False + + def require_upstream(self) -> None: + """Raise unless :meth:`probe` reports this plane can be served. + + The default probe means the upstream package is importable; + overrides are honored. + + Raises: + RuntimeError: when :meth:`probe` is false, naming the + ``upstream`` distribution (or :attr:`name`) and + ``pip install ...``. + """ + if self.probe(): + return + target = self.upstream or self.name + raise RuntimeError( + f"the {self.name!r} plane requires the {target!r} package. " + f"Install with: pip install {target}" + ) + + # -- registration -------------------------------------------------- + + def tool_specs(self) -> Iterator[ToolSpec]: + """Every declared tool, base classes first, in declaration order. + + Returns: + :class:`ToolSpec` values, base classes first, in class-body + order. Redefining the same attribute on a subclass replaces that + spec; two different attributes that claim one wire name are both + yielded here and rejected later by :meth:`register`. + """ + found: dict[str, ToolSpec] = {} + for klass in reversed(type(self).__mro__): + for attribute, value in vars(klass).items(): + marker = getattr(value, _MARKER, None) + if marker is None: + continue + wire_name, annotations = marker + found[attribute] = ToolSpec( + name=wire_name, annotations=annotations, attribute=attribute + ) + return iter(found.values()) + + def register(self, mcp: FastMCP) -> None: + """Attach this plane's tools to its server. + + Bound methods are handed to FastMCP directly: ``self`` is already + applied, so it never reaches the parameter list FastMCP publishes to + the MCP client, and the docstring the MCP client reads is the one on + the method. + + Args: + mcp: FastMCP server this plane attaches tools to (the server + whose name is :attr:`name`). + + Raises: + RuntimeError: :meth:`require_upstream` failed (``probe()`` is + false). + ValueError: two methods claim the same wire name. Overriding by + *attribute* is intended — a subclass redefining a tool + replaces it — but two distinct methods claiming one name is + one tool shadowing another, and which survives would depend + on method resolution order (the class's base-class chain). + """ + self.require_upstream() + claimed: dict[str, str] = {} + for spec in self.tool_specs(): + previous = claimed.get(spec.name) + if previous is not None: + raise ValueError( + f"{type(self).__name__} declares the tool name " + f"{spec.name!r} twice: {previous}() and {spec.attribute}()" + ) + claimed[spec.name] = spec.attribute + mcp.tool(name=spec.name, annotations=spec.annotations)( + getattr(self, spec.attribute) + ) + + +__all__ = [ + "APPEND_WRITE", + "IDEMPOTENT_WRITE", + "LOCAL_MUTATION", + "MUTATION", + "Provider", + "ProviderBase", + "READ_ONLY", + "READ_REMOTE", + "ToolSpec", + "tool", +] diff --git a/src/molmcp/provider_worker/__init__.py b/src/molmcp/provider_worker/__init__.py new file mode 100644 index 0000000..f8a6948 --- /dev/null +++ b/src/molmcp/provider_worker/__init__.py @@ -0,0 +1,46 @@ +"""Subprocess-hosted provider plane — lazy façade. + +``WorkerProvider`` lives in the sibling ``worker`` module, which is free to +import FastMCP and the supervisor/proxy machinery. This package body must not — +the worker child process imports ``molmcp.provider_worker.protocol``, and every +statement executed here is a statement the child pays for. Resolving the one +public name through PEP 562 keeps the child's ``sys.modules`` free of FastMCP, +so ``child.py``'s isolation assertion measures a real leak rather than the +import that always happens. +""" + +from __future__ import annotations + +__all__ = ["WorkerProvider"] + + +def __getattr__(name: str) -> object: + """Resolve ``WorkerProvider`` from the sibling ``worker`` module. + + Args: + name: Attribute requested on the ``molmcp.provider_worker`` package. + + Returns: + The ``WorkerProvider`` class, cached into the module globals so the + import happens at most once. + + Raises: + AttributeError: For any other name, which is also what lets CPython + fall back to importing a submodule such as ``protocol``. + """ + if name != "WorkerProvider": + raise AttributeError(f"module {__name__!r} has no attribute {name!r}") + from .worker import WorkerProvider + + globals()[name] = WorkerProvider + return WorkerProvider + + +def __dir__() -> list[str]: + """List the public surface plus whatever has already been resolved. + + Returns: + Sorted attribute names, including ``WorkerProvider`` whether or not + the ``worker`` module has been imported yet. + """ + return sorted(set(globals()) | set(__all__)) diff --git a/src/molmcp/provider_worker/child.py b/src/molmcp/provider_worker/child.py new file mode 100644 index 0000000..a7d2c0e --- /dev/null +++ b/src/molmcp/provider_worker/child.py @@ -0,0 +1,338 @@ +"""The worker child — one provider plane in its own process, speaking duplex v1. + +A supervisor launches this file *by path* +(``python -P child.py --entrypoint package.module:ClassName --path ``), +never as a module of an installed package: the plane's code may live anywhere +on disk, and ``--path`` is the only root it is imported from. ``-P`` keeps this +script's own directory off ``sys.path``, so nothing sitting next to it can +shadow that root. + +The launch vector is the whole configuration. Nothing here reads the +environment: two planes started by two different clients would otherwise +disagree about a setting no ``molmcp config list`` could report. + +*Duplex v1* is the wire format both sides speak: one JSON object per line +(NDJSON, newline-delimited JSON) in each direction, frozen in +:mod:`molmcp.provider_worker.protocol`. + +The child imports the real provider base and the real wire format, and nothing +else of molmcp — no server library, no composition, no stub standing in for +either. It answers three things: a ``hello`` catalog of signature facts, one +``result`` or ``error`` per ``invoke``, and exit 0 on ``shutdown``. Turning +those facts into MCP tools is the parent's half of the job, so the schema a +client finally sees is built by the same machinery an in-process plane uses. + +Isolation is asserted, not assumed. If the MCP server library or molmcp's own +composition module is resident once the plane has been constructed, the child +reports which modules leaked and exits non-zero *without* saying hello: a +worker that drags the server library into its own process has bought nothing, +and failing loudly at startup is cheaper than discovering it in production. +""" + +from __future__ import annotations + +import argparse +import importlib +import inspect +import sys +from collections.abc import Callable, Mapping, Sequence +from typing import IO, TYPE_CHECKING, Any + +if TYPE_CHECKING: + from molmcp.provider_sdk import ProviderBase + +#: The MCP server library this process exists to stay out of. Named as a +#: string because the child must be able to detect it without importing it. +_SERVER_LIBRARY = "fastmcp" + +#: molmcp's own composition module. Loading it here would mean the child had +#: gone through the server rather than straight to the plane. +_SERVER_MODULE = "molmcp.server" + +#: Wire id for a frame that answers no call — a startup failure, or a line +#: that could not be decoded far enough to carry an id. +_NO_CALL = "" + +#: Exit status of a child that could not prove its own isolation. +_ISOLATION_FAILURE = 2 + + +def _parse_args(argv: list[str] | None) -> argparse.Namespace: + """Read the launch vector. + + Args: + argv: Arguments to parse, or None to read ``sys.argv[1:]``. + + Returns: + A namespace carrying ``entrypoint`` and ``path``. Both are required: + a default for either would let a mislaunched child serve a plane + nobody asked for. + """ + parser = argparse.ArgumentParser( + prog="child.py", + description="Serve one provider plane over duplex v1 on stdio.", + ) + parser.add_argument( + "--entrypoint", + required=True, + help="Provider class to serve, as 'package.module:ClassName'.", + ) + parser.add_argument( + "--path", + required=True, + help="Directory prepended to sys.path — the only root the entrypoint " + "is imported from.", + ) + return parser.parse_args(argv) + + +def _instantiate(entrypoint: str, base: type[ProviderBase]) -> ProviderBase: + """Import the module the entrypoint names and construct its class. + + Nothing below is caught. Every one of these failures happens before the + child says hello, so the parent sees a single symptom — a handshake that + never completed — and the child's traceback on the inherited stderr says + which failure it was. + + Args: + entrypoint: ``package.module:ClassName``. + base: Provider base class the named class must derive from. + + Returns: + A new instance of that class. + + Raises: + ValueError: If *entrypoint* is not two non-empty parts, or names + something that is not a *base* subclass. Both are broken launch + vectors. + ImportError: If the module cannot be imported from the ``--path`` + root — the usual shape of "this checkout does not run here". + AttributeError: If that module has no attribute named ``ClassName``. + """ + module_name, separator, class_name = entrypoint.partition(":") + if not (separator and module_name and class_name): + raise ValueError( + f"--entrypoint must be 'package.module:ClassName', got {entrypoint!r}" + ) + candidate = getattr(importlib.import_module(module_name), class_name) + if not (isinstance(candidate, type) and issubclass(candidate, base)): + raise ValueError(f"{entrypoint} is not a {base.__name__} subclass") + return candidate() + + +def _leaked_modules() -> list[str]: + """Loaded modules that betray the isolation this process exists for. + + Returns: + Sorted names of every resident module belonging to the MCP server + library or to molmcp's composition layer. Empty means the child got + to its plane without going through a server. + """ + return [ + name + for name in sorted(sys.modules) + if name in (_SERVER_LIBRARY, _SERVER_MODULE) + or name.startswith(f"{_SERVER_LIBRARY}.") + ] + + +def _tool_facts(instance: ProviderBase) -> list[dict[str, Any]]: + """Describe every tool the plane declares, as hello catalog entries. + + Each signature is read off the *bound* method, so ``self`` never reaches + the parent. Facts are all that travel: names, kinds, annotations and + defaults. The parent rebuilds a callable from them, and its MCP server + derives the schema — the child never spells one out. + + Args: + instance: The provider this child serves. + + Returns: + One entry per declared tool, in declaration order, each with + ``name``, ``attribute``, ``doc``, ``annotations`` and ``parameters``. + """ + from molmcp.provider_worker.protocol import ANNOTATION_KEYS, signature_facts + + facts: list[dict[str, Any]] = [] + for spec in instance.tool_specs(): + method = getattr(instance, spec.attribute) + facts.append( + { + "name": spec.name, + "attribute": spec.attribute, + "doc": method.__doc__ or "", + "annotations": { + key: bool(getattr(spec.annotations, key)) for key in ANNOTATION_KEYS + }, + "parameters": signature_facts(inspect.signature(method)), + } + ) + return facts + + +def _tool_methods( + instance: ProviderBase, facts: Sequence[Mapping[str, Any]] +) -> dict[str, Callable[..., Any]]: + """Bind each catalog entry to the method that implements it. + + Args: + instance: The provider this child serves. + facts: The catalog entries sent in hello. + + Returns: + Bound methods by *bare* tool name — the same name the parent invokes + by, never a namespaced one. + """ + return {fact["name"]: getattr(instance, fact["attribute"]) for fact in facts} + + +def _write(stream: IO[str], line: str) -> None: + """Write one NDJSON line and flush it. + + A frame the parent cannot read yet is a frame it will block on, so every + write is flushed rather than left to the pipe's buffer. + + Args: + stream: Where the answer goes. + line: One complete NDJSON line, newline included. + """ + stream.write(line) + stream.flush() + + +def _render(exc: BaseException) -> str: + """Render an exception for the wire. + + Args: + exc: The failure to report. + + Returns: + ``"TypeName: message"``. The exception object cannot cross a pipe and + the two processes share no traceback, so this text is the whole + diagnosis the parent gets to re-raise. + """ + return f"{type(exc).__name__}: {exc}" + + +def _call(frame: Mapping[str, Any], methods: Mapping[str, Callable[..., Any]]) -> Any: + """Run the tool an ``invoke`` frame names. + + Args: + frame: A decoded frame, expected to be an ``invoke``. + methods: Bound methods by bare tool name. + + Returns: + Whatever the tool returned, to travel as the ``result`` value. + + Raises: + ValueError: If the frame is not an ``invoke``. + LookupError: If no tool answers to that name. The parent bound its + tools from this child's own hello, so the message lists what is + actually offered. + Exception: Whatever the tool itself raises. + """ + if frame["type"] != "invoke": + raise ValueError(f"expected an invoke frame, got {frame['type']!r}") + name = frame["name"] + if name not in methods: + raise LookupError( + f"no tool named {name!r}; this plane offers {sorted(methods)}" + ) + return methods[name](**frame["args"]) + + +def _serve( + methods: Mapping[str, Callable[..., Any]], stdin: IO[str], stdout: IO[str] +) -> int: + """Answer frames until the parent says shutdown or closes the pipe. + + A failing call is an ``error`` frame, never an exit: one bad call must + not cost the parent its worker. The same holds for a line that is not a + frame at all — it is reported against no call id and the loop goes on. + + Args: + methods: Bound methods by bare tool name. + stdin: Stream the parent's frames arrive on. + stdout: Stream every answer is written and flushed to. + + Returns: + 0 — reached on a ``shutdown`` frame, or on end of input, which is the + parent having closed the pipe. + """ + from molmcp.provider_worker.protocol import ( + ProtocolError, + decode, + encode_error, + encode_result, + ) + + for line in iter(stdin.readline, ""): + try: + frame = decode(line) + except ProtocolError as exc: + _write(stdout, encode_error(call_id=_NO_CALL, error=_render(exc))) + continue + if frame["type"] == "shutdown": + return 0 + call_id = frame.get("id", _NO_CALL) + try: + value = _call(frame, methods) + except Exception as exc: + _write(stdout, encode_error(call_id=call_id, error=_render(exc))) + continue + _write(stdout, encode_result(call_id=call_id, value=value)) + return 0 + + +def main(argv: list[str] | None = None) -> int: + """Serve one provider plane on stdio. + + Args: + argv: Arguments after the script path, or None to read + ``sys.argv[1:]``. + + Returns: + 0 after a clean shutdown or a closed pipe; 2 when the isolation + assertion fails, in which case no hello was ever sent — the parent + gets an ``error`` frame carrying the names of the leaked modules. + + Raises: + SystemExit: ``--entrypoint`` or ``--path`` is missing or unparsable. + argparse prints usage to stderr and raises this itself; the child + never reaches the plane. + Exception: Whatever loading the entrypoint raises — see + :func:`_instantiate`. Deliberately not caught: the child ends + without a hello, and the parent reports the handshake failure. + """ + args = _parse_args(argv) + sys.path.insert(0, args.path) + + try: + from molmcp.provider_sdk import ProviderBase + except ImportError: # molmcp predating the public SDK module + from molmcp.providers.base import ProviderBase + from molmcp.provider_worker.protocol import encode_error, encode_hello + + instance = _instantiate(args.entrypoint, ProviderBase) + + leaked = _leaked_modules() + if leaked: + _write( + sys.stdout, + encode_error( + call_id=_NO_CALL, + error=( + f"worker isolation broken: serving {args.entrypoint} " + f"loaded {', '.join(leaked)} in the child process" + ), + ), + ) + return _ISOLATION_FAILURE + + facts = _tool_facts(instance) + _write(sys.stdout, encode_hello(name=instance.name, tools=facts)) + return _serve(_tool_methods(instance, facts), sys.stdin, sys.stdout) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/molmcp/provider_worker/protocol.py b/src/molmcp/provider_worker/protocol.py new file mode 100644 index 0000000..45bf2dd --- /dev/null +++ b/src/molmcp/provider_worker/protocol.py @@ -0,0 +1,397 @@ +"""Duplex v1 — the NDJSON wire format between a provider child and its parent. + +A worker provider runs a plain :class:`~molmcp.provider_sdk.ProviderBase` +subclass in a child process and proxies its tools onto a FastMCP server in the +parent. This module is the one place that wire format is written down, and it +is imported by the child, so it depends on the standard library only: every +import here is an import the child pays for before it can say hello. + +One frame is one line of JSON followed by ``"\\n"`` (NDJSON). Every frame +carries ``"type"`` and ``"protocol"``; a line missing either is refused rather +than guessed at. + +Frame table (v1) +---------------- + +=========== ================ ========================================= +type direction payload +=========== ================ ========================================= +``hello`` child -> parent ``name`` (the plane id) and ``tools``: the + catalog, one entry per declared tool, each + with ``name`` (the bare tool name), + ``attribute`` (the method that implements + it), ``doc`` (that method's docstring), + ``annotations`` (the four booleans in + :data:`ANNOTATION_KEYS`) and + ``parameters`` (signature facts, see + :func:`signature_facts`). +``invoke`` parent -> child ``id`` (the call id), ``name`` (bare tool + name) and ``args`` (keyword arguments). +``result`` child -> parent ``id``, ``ok`` (true) and ``value`` — what + the tool returned. +``error`` child -> parent ``id``, ``ok`` (false) and ``error`` — the + failure rendered as a string. +``shutdown`` parent -> child nothing. The child exits; the parent waits + and may terminate it if it does not. +=========== ================ ========================================= + +The call id travels under the wire key ``"id"``; the Python keyword argument +is ``call_id``, so no function here shadows the builtin. + +Version mismatch +---------------- + +Either side reading ``protocol != 1`` fails: :func:`decode` raises +:class:`ProtocolError` and the parent shuts the child down and raises. There +is no negotiation and no downgrade — a silent downgrade would let a child +built against a different frame table answer as if it agreed. + +Signature facts, not JSON Schema +-------------------------------- + +The child sends what it knows — parameter names, kinds, annotations and +defaults — and never a JSON Schema. The parent rebuilds an +:class:`inspect.Signature` from those facts and hands FastMCP a callable; +FastMCP stays the only producer of JSON Schema, so the schema a client sees +comes from the same machinery an in-process provider would have used. +""" + +from __future__ import annotations + +import inspect +import json +from collections.abc import Mapping, Sequence +from typing import Any + +#: The wire format this module speaks. Bumped only by a spec that changes the +#: frame table; both sides refuse anything else. +PROTOCOL_VERSION: int = 1 + +#: Every frame type duplex v1 admits. :func:`decode` refuses the rest, so a +#: typo in a ``type`` is an error at the boundary rather than a silent no-op. +MESSAGE_TYPES: frozenset[str] = frozenset( + {"hello", "invoke", "result", "error", "shutdown"} +) + +#: The MCP ``ToolAnnotations`` hints a hello catalog carries, in the order the +#: catalog writes them. Ordered, because it is also the order a reader reasons +#: about a tool in: what it reads, what it destroys, whether repeating it is +#: safe, and how far it reaches. +ANNOTATION_KEYS: tuple[str, str, str, str] = ( + "read_only_hint", + "destructive_hint", + "idempotent_hint", + "open_world_hint", +) + +#: Annotation names that survive the round trip as real objects. Anything else +#: stays the string the child sent: the parent cannot import a provider's own +#: classes, and a string annotation is honest about that. +_BUILTIN_ANNOTATIONS: dict[str, type | None] = { + "str": str, + "int": int, + "float": float, + "bool": bool, + "list": list, + "dict": dict, + "None": None, +} + +#: ``inspect.Parameter`` kinds by attribute name — the vocabulary +#: :func:`signature_facts` writes and :func:`rebuild_signature` reads. +_PARAMETER_KINDS = { + kind.name: kind + for kind in ( + inspect.Parameter.POSITIONAL_ONLY, + inspect.Parameter.POSITIONAL_OR_KEYWORD, + inspect.Parameter.VAR_POSITIONAL, + inspect.Parameter.KEYWORD_ONLY, + inspect.Parameter.VAR_KEYWORD, + ) +} + +#: How much of an offending line an error message quotes. Long enough to +#: recognise the frame, short enough not to bury the reason in it. +_EXCERPT = 120 + + +class ProtocolError(RuntimeError): + """Something duplex v1 cannot accept. + + :func:`decode` raises it for a line that is not JSON, is not a JSON + object, lacks ``type`` or ``protocol``, names a type outside + :data:`MESSAGE_TYPES`, or declares a protocol other than + :data:`PROTOCOL_VERSION`. :func:`rebuild_signature` raises it for a + catalog entry whose ``kind`` is not an :class:`inspect.Parameter` kind. + The message names which of those it was: the two processes share no + traceback, so the text is the whole diagnosis. + + It subclasses ``RuntimeError``, so a caller that only wants "talking to + the worker went wrong" can catch that one type. + """ + + +def _excerpt(line: str) -> str: + """Trim ``line`` to a quotable length for an error message. + + Args: + line: The raw line that failed to decode. + + Returns: + The line stripped of surrounding whitespace, truncated with an + ellipsis when it is longer than :data:`_EXCERPT`. + """ + text = line.strip() + if len(text) <= _EXCERPT: + return text + return f"{text[:_EXCERPT]}..." + + +def _line(payload: dict[str, Any]) -> str: + """Render one frame as a single NDJSON line. + + Args: + payload: Frame body. ``type`` must already be set; ``protocol`` is + added here so no encoder can forget it. + + Returns: + A single line of JSON ending in a newline. ``json.dumps`` escapes + every newline inside the payload, so the result is always exactly + one line. + """ + return json.dumps({**payload, "protocol": PROTOCOL_VERSION}) + "\n" + + +def encode_hello(*, name: str, tools: Sequence[Mapping[str, Any]]) -> str: + """Encode the child's opening catalog. + + ``hello`` is the only place tools are declared. The parent builds its + FastMCP tools from this frame and asks the child nothing else about them. + + Args: + name: The plane id the child provider answers to. + tools: One catalog entry per declared tool — ``name``, ``attribute``, + ``doc``, ``annotations`` and ``parameters``. + + Returns: + One NDJSON line carrying ``type``, ``protocol``, ``name`` and + ``tools``. + """ + return _line({"type": "hello", "name": name, "tools": [dict(t) for t in tools]}) + + +def encode_invoke(*, call_id: str, name: str, args: Mapping[str, Any]) -> str: + """Encode a parent-to-child tool call. + + Args: + call_id: Identifier the matching ``result`` or ``error`` echoes back. + It travels under the wire key ``"id"``. + name: Bare tool name, as it appeared in the hello catalog. + args: Keyword arguments for the call. + + Returns: + One NDJSON line carrying ``type``, ``protocol``, ``id``, ``name`` and + ``args``. + """ + return _line({"type": "invoke", "id": call_id, "name": name, "args": dict(args)}) + + +def encode_result(*, call_id: str, value: Any) -> str: + """Encode a successful call's return value. + + Args: + call_id: The ``id`` of the ``invoke`` being answered. + value: What the tool returned. Must be JSON-serializable — a provider + tool returns MCP payloads, so this is the same constraint MCP + already puts on it. + + Returns: + One NDJSON line carrying ``type``, ``protocol``, ``id``, ``ok`` (true) + and ``value``. + """ + return _line({"type": "result", "id": call_id, "ok": True, "value": value}) + + +def encode_error(*, call_id: str, error: str) -> str: + """Encode a failed call. + + Args: + call_id: The ``id`` of the ``invoke`` being answered. + error: The failure as a string. The exception object cannot cross a + pipe, so the child renders it and the parent re-raises the text. + + Returns: + One NDJSON line carrying ``type``, ``protocol``, ``id``, ``ok`` + (false) and ``error``. + """ + return _line({"type": "error", "id": call_id, "ok": False, "error": error}) + + +def encode_shutdown() -> str: + """Encode the parent's request that the child exit. + + Returns: + One NDJSON line carrying ``type`` and ``protocol`` and nothing else — + the frame is the whole message. + """ + return _line({"type": "shutdown"}) + + +def decode(line: str) -> dict[str, Any]: + """Parse and validate one NDJSON frame. + + Args: + line: A single line read from the pipe, newline included or not. + + Returns: + The parsed frame, with ``type`` and ``protocol`` known good. + + Raises: + ProtocolError: If the line is not JSON, is not a JSON object, is + missing ``type`` or ``protocol``, names a type outside + :data:`MESSAGE_TYPES`, or declares a protocol other than + :data:`PROTOCOL_VERSION`. + """ + try: + frame = json.loads(line) + except ValueError as exc: + raise ProtocolError(f"frame is not JSON: {_excerpt(line)!r} ({exc})") from exc + if not isinstance(frame, dict): + raise ProtocolError( + f"frame is not a JSON object but {type(frame).__name__}: {_excerpt(line)!r}" + ) + if "type" not in frame: + raise ProtocolError(f'frame is missing "type": {_excerpt(line)!r}') + kind = frame["type"] + if kind not in MESSAGE_TYPES: + raise ProtocolError( + f"unknown frame type {kind!r}; duplex v1 speaks {sorted(MESSAGE_TYPES)}" + ) + if "protocol" not in frame: + raise ProtocolError(f'frame is missing "protocol": {_excerpt(line)!r}') + version = frame["protocol"] + if version != PROTOCOL_VERSION: + raise ProtocolError( + f"protocol version mismatch: frame declares {version!r}, this side " + f"speaks {PROTOCOL_VERSION}" + ) + return frame + + +def _is_json_safe(value: Any) -> bool: + """Report whether ``value`` can cross the wire as JSON. + + Args: + value: A parameter default taken from a live signature. + + Returns: + ``True`` when :func:`json.dumps` accepts it, ``False`` otherwise. + """ + try: + json.dumps(value) + except (TypeError, ValueError): + return False + return True + + +def _annotation_name(annotation: Any) -> str: + """Render one parameter annotation as a string. + + Args: + annotation: The annotation from an :class:`inspect.Parameter`. + + Returns: + ``""`` for an empty annotation, the type's ``__name__`` when the + annotation is a type, and ``str(annotation)`` for everything else — + a typing construct or a forward reference the parent cannot resolve. + """ + if annotation is inspect.Parameter.empty: + return "" + if isinstance(annotation, type): + name = getattr(annotation, "__name__", None) + if isinstance(name, str): + return name + return str(annotation) + + +def signature_facts(signature: inspect.Signature) -> list[dict[str, Any]]: + """Describe a signature as a list of per-parameter facts. + + This is deliberately not a JSON Schema. The child states what its method + takes; the parent rebuilds a callable and lets FastMCP derive the schema, + so a proxied tool and an in-process one are described by the same code. + + Args: + signature: A *bound* method signature — ``self`` is already gone. + + Returns: + One dict per parameter, in declaration order, with ``name``, ``kind`` + (the :class:`inspect.Parameter` attribute name, e.g. + ``"POSITIONAL_OR_KEYWORD"``), ``annotation`` (see + :func:`_annotation_name`) and ``has_default``. A ``default`` key is + present only when there is a default *and* it is JSON-serializable; + an unserializable default is omitted rather than approximated. + """ + facts: list[dict[str, Any]] = [] + for parameter in signature.parameters.values(): + has_default = parameter.default is not inspect.Parameter.empty + fact: dict[str, Any] = { + "name": parameter.name, + "kind": parameter.kind.name, + "annotation": _annotation_name(parameter.annotation), + "has_default": has_default, + } + if has_default and _is_json_safe(parameter.default): + fact["default"] = parameter.default + facts.append(fact) + return facts + + +def rebuild_signature(facts: Sequence[Mapping[str, Any]]) -> inspect.Signature: + """Rebuild an :class:`inspect.Signature` from :func:`signature_facts`. + + Builtin annotation names come back as the type objects, so FastMCP sees + ``str`` rather than ``"str"``; anything else stays a string, which FastMCP + treats as an unresolved annotation instead of guessing at a class the + parent never imported. + + Args: + facts: The ``parameters`` list from one hello catalog entry. + + Returns: + A signature with no return annotation. The wire format carries no + return type — a hello catalog entry has no field for one — and does + not need to: FastMCP builds the structured result from the value the + tool actually returned, so a proxied tool that returns a dict reaches + the client as structured content exactly as an in-process one does. + + Raises: + ProtocolError: If a fact names a parameter kind that is not an + :class:`inspect.Parameter` kind. + """ + parameters: list[inspect.Parameter] = [] + for fact in facts: + kind_name = fact["kind"] + kind = _PARAMETER_KINDS.get(kind_name) + if kind is None: + raise ProtocolError( + f"unknown parameter kind {kind_name!r} for {fact['name']!r}; " + f"expected one of {sorted(_PARAMETER_KINDS)}" + ) + annotation_name = fact["annotation"] + if annotation_name == "": + annotation: Any = inspect.Parameter.empty + elif annotation_name in _BUILTIN_ANNOTATIONS: + annotation = _BUILTIN_ANNOTATIONS[annotation_name] + else: + annotation = annotation_name + # A default the child could not serialize left no ``default`` key, so + # the parameter comes back required. Better a caller that must pass a + # value than a parent that invents one the provider never chose. + default = fact.get("default", inspect.Parameter.empty) + parameters.append( + inspect.Parameter( + fact["name"], kind, default=default, annotation=annotation + ) + ) + return inspect.Signature(parameters) diff --git a/src/molmcp/provider_worker/proxy.py b/src/molmcp/provider_worker/proxy.py new file mode 100644 index 0000000..d9458d3 --- /dev/null +++ b/src/molmcp/provider_worker/proxy.py @@ -0,0 +1,111 @@ +"""Proxy — a child's hello catalog becomes published FastMCP tools. + +The child sends signature *facts*, never a schema. This module turns each fact +list back into an :class:`inspect.Signature`, hangs it on a callable, and hands +that callable to ``mcp.tool(...)`` — the same call an in-process provider's +``register`` makes. FastMCP therefore stays the only producer of JSON Schema in +the system, so a proxied tool and a local one are described to a client by the +same machinery rather than by two descriptions that have to be kept in step. + +The docstring a client reads is the child's method docstring, and the +``ToolAnnotations`` are the child's declared ones: crossing a process boundary +must not quietly downgrade what a caller is told before it confirms a call. + +Nothing here knows about subprocesses. ``invoke`` is a plain callable, so this +module is exercised with a recording stub and the Supervisor is exercised +separately. +""" + +from __future__ import annotations + +import inspect +from collections.abc import Callable, Mapping, Sequence +from typing import TYPE_CHECKING, Any + +from mcp.types import ToolAnnotations + +from .protocol import rebuild_signature + +if TYPE_CHECKING: + from fastmcp import FastMCP + + +def _build_call( + fact: Mapping[str, Any], + invoke: Callable[[str, dict[str, Any]], Any], +) -> Callable[..., Any]: + """Build the parent-side callable standing in for one child tool. + + Arguments are bound against the rebuilt signature before they cross the + pipe, so a bad call fails here — with the child's own parameter names in + the message — rather than as an ``error`` frame from a process the caller + cannot see. Defaults are applied for the same reason: the child receives + the call the signature says it will, whether or not the client spelled + every argument out. + + Args: + fact: One hello catalog entry — ``name``, ``doc`` and ``parameters``. + invoke: How a call reaches the child: bare tool name and arguments. + + Returns: + A callable carrying the child's tool name, docstring, signature and + per-parameter type annotations — what FastMCP reads off a function + when it publishes it. The MCP ``ToolAnnotations`` (the hints a client + uses to decide whether to confirm a call) are *not* on the callable; + :func:`bind_tools` passes those to ``mcp.tool`` itself. + """ + name: str = fact["name"] + signature = rebuild_signature(fact["parameters"]) + + def call(*args: object, **kwargs: object) -> Any: + bound = signature.bind(*args, **kwargs) + bound.apply_defaults() + return invoke(name, dict(bound.arguments)) + + call.__name__ = name + call.__doc__ = fact["doc"] + call.__signature__ = signature # type: ignore[attr-defined] + call.__annotations__ = { + parameter.name: parameter.annotation + for parameter in signature.parameters.values() + if parameter.annotation is not inspect.Parameter.empty + } + return call + + +def bind_tools( + mcp: FastMCP, + hello: Mapping[str, Any], + invoke: Callable[[str, dict[str, Any]], Any], +) -> list[str]: + """Publish every tool a child declared onto its FastMCP server. + + Names are registered **bare** (``echo``, never ``echo_echo``): a composed + core adds the namespace when it mounts the plane, and a plane that + prefixed its own names would be namespaced twice. + + Args: + mcp: The server for this plane — the one whose name is the plane id. + hello: The child's greeting; only its ``tools`` list is read. + invoke: How a published tool reaches the child. Bound at publish time, + so the tool holds the Supervisor rather than looking one up. + + Returns: + The bare names bound, in the order the child declared them. + + Raises: + ProtocolError: Raised by + :func:`~molmcp.provider_worker.protocol.rebuild_signature` when a + catalog entry names a parameter kind that is not an + :class:`inspect.Parameter` kind. Tools declared before the bad + entry are already bound when this happens, which is why the + caller's failure path reaps the child instead of carrying on with + a half-bound plane. + """ + tools: Sequence[Mapping[str, Any]] = hello["tools"] + bound: list[str] = [] + for fact in tools: + annotations = ToolAnnotations(**fact["annotations"]) + mcp.tool(name=fact["name"], annotations=annotations)(_build_call(fact, invoke)) + bound.append(fact["name"]) + return bound diff --git a/src/molmcp/provider_worker/supervisor.py b/src/molmcp/provider_worker/supervisor.py new file mode 100644 index 0000000..fe5ff72 --- /dev/null +++ b/src/molmcp/provider_worker/supervisor.py @@ -0,0 +1,337 @@ +"""Supervisor — the parent's single owner of one worker plane's child process. + +Exactly one object in the parent holds the child: it launches it, reads its +``hello``, turns each tool call into an ``invoke`` frame, and reaps it. Nothing +else touches the pipes, so "is the child alive, and who is allowed to end it?" +has one answer instead of one per caller. + +Those frames are *duplex v1*: one JSON object per line (NDJSON, +newline-delimited JSON) in each direction, frozen in +:mod:`molmcp.provider_worker.protocol`. + +The launch is a **path launch** — ``python -P child.py --entrypoint ... --path +...`` — never ``python -m``. A worker plane's code lives in a checkout, not in +an installed distribution, so there is no module path to name it by; ``-P`` +additionally keeps the script's own directory off ``sys.path`` so nothing +sitting beside ``child.py`` can shadow the checkout that ``--path`` names. + +The launch vector is the entire configuration. This module reads nothing from +the surrounding process: a setting that lives only in one shell cannot be +reported by ``molmcp config list``, and two planes started by two different +clients would silently disagree about it. + +The subprocess itself is injected (``spawn=``), so the wire behaviour above can +be proved against a fake pair of streams without ever forking. +""" + +from __future__ import annotations + +import itertools +import logging +import subprocess +import sys +from collections.abc import Callable, Iterator, Mapping +from pathlib import Path +from typing import Any, Protocol + +from .protocol import ProtocolError, decode, encode_invoke, encode_shutdown + +logger = logging.getLogger(__name__) + +#: The script every Supervisor launches. Resolved from this module's own +#: location, so a checkout, a wheel and an editable install all find the child +#: that matches the protocol module they are about to speak. +CHILD_SCRIPT: Path = Path(__file__).with_name("child.py") + +#: How long a child gets to exit on its own after being asked to. Long enough +#: for an in-flight call to finish, short enough that shutting a server down +#: does not look like a hang. +_EXIT_TIMEOUT: float = 5.0 + +#: How long a *terminated* child gets before it is written off. A process that +#: ignores SIGTERM this long is reported rather than waited on forever: a +#: parent blocked in teardown is worse than a leaked child it has named. +_TERMINATE_TIMEOUT: float = 5.0 + + +class _ChildStdin(Protocol): + """The write half of the pipe, as this module uses it.""" + + def write(self, data: str, /) -> int: ... + + def flush(self) -> None: ... + + def close(self) -> None: ... + + +class _ChildStdout(Protocol): + """The read half of the pipe: one NDJSON line at a time, ``""`` at EOF.""" + + def readline(self) -> str: ... + + def close(self) -> None: ... + + +class _ChildProcess(Protocol): + """What a spawned child must expose — a narrow slice of ``Popen``. + + Both streams are required, not optional: the only spawn this module ships + opens them as pipes, and a seam that hands back a child it cannot talk to + has not spawned anything useful. + + The two streams plus ``wait`` and ``terminate`` are what this module + calls. ``poll`` — "has it exited yet, and with what status?" — is never + called here; it is part of the seam's contract because it is how a caller + holding the spawned object asks whether the child is still alive. + """ + + stdin: _ChildStdin + stdout: _ChildStdout + + def wait(self, timeout: float | None = None) -> int: ... + + def terminate(self) -> None: ... + + def poll(self) -> int | None: ... + + +def _default_spawn(argv: list[str]) -> _ChildProcess: + """Launch the child for real, over text pipes. + + No ``env=`` is passed: the child inherits the parent's environment + unchanged. Configuration travels in *argv*, which the parent can print and + a reader can reproduce. + + Args: + argv: The launch vector, as built by :attr:`Supervisor.argv`. + + Returns: + The running child, line-buffered so a flushed frame arrives whole. + """ + return subprocess.Popen( + argv, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + text=True, + bufsize=1, + ) + + +class Supervisor: + """One child process serving one provider plane, over duplex v1. + + Args: + entrypoint: The provider class the child serves, as + ``package.module:ClassName``. + path: Directory the child imports that module from — the checkout + root, passed through to ``--path``. + spawn: Seam that turns a launch vector into a running process. Defaults + to a real ``subprocess.Popen`` over text pipes; tests pass a fake + pair of streams so the frame handling here is proved without a + fork. + """ + + def __init__( + self, + *, + entrypoint: str, + path: str | Path, + spawn: Callable[[list[str]], _ChildProcess] | None = None, + ) -> None: + self._entrypoint = entrypoint + self._path = path + self._spawn = spawn if spawn is not None else _default_spawn + self._process: _ChildProcess | None = None + self._stopped = False + self._call_ids: Iterator[int] = itertools.count(1) + + @property + def argv(self) -> list[str]: + """The launch vector, and the whole of the child's configuration. + + Returns: + ``[sys.executable, "-P", , "--entrypoint", ..., + "--path", ...]``. Never contains ``-m``: the plane is a checkout + on disk, not an installed module. Reading this property starts + nothing. + """ + return [ + sys.executable, + "-P", + str(CHILD_SCRIPT), + "--entrypoint", + self._entrypoint, + "--path", + str(self._path), + ] + + def start(self) -> dict[str, Any]: + """Launch the child and read the catalog it greets with. + + Returns: + The decoded ``hello`` frame — the plane id and one entry per tool. + It is the only tool declaration there is; the parent asks the child + nothing else about them. + + Raises: + RuntimeError: The first line was not a valid duplex v1 ``hello``: + unreadable, a mismatched protocol version, or some other frame + type. The child is shut down first — a child that cannot be + talked to is still a process, and leaving it running to report + a handshake failure trades one problem for two. + """ + process = self._spawn(self.argv) + self._process = process + + line = process.stdout.readline() + try: + frame = decode(line) + except ProtocolError as exc: + self.shutdown() + raise RuntimeError( + f"the worker child for {self._entrypoint!r} did not greet in " + f"duplex v1: {exc}" + ) from exc + + if frame["type"] != "hello": + self.shutdown() + raise RuntimeError( + f"the worker child for {self._entrypoint!r} opened with a " + f"{frame['type']!r} frame; duplex v1 opens with 'hello'" + ) + return frame + + def invoke(self, name: str, args: Mapping[str, Any]) -> Any: + """Call one tool in the child and wait for its answer. + + Calls are strictly one at a time: one ``invoke`` written, one frame + read back, so the reply can only belong to the call just made. Duplex + v1 still puts a call id on both frames. This method does not compare + them — with a single outstanding call there is nothing to disambiguate + — but the id is on the wire, so a recorded exchange can be paired up + afterwards without counting lines. + + Args: + name: Bare tool name, as it appeared in the hello catalog. + args: Keyword arguments for the call. + + Returns: + Whatever the tool returned, decoded from the ``result`` frame. + + Raises: + RuntimeError: The child has not been started, has already been + shut down, has closed the pipe, answered with an ``error`` + frame (whose text is re-raised verbatim — the two processes + share no traceback), or answered with a frame that does not + answer a call at all. + ProtocolError: The reply was not a valid duplex v1 frame. It is + itself a ``RuntimeError``, so one ``except RuntimeError`` + covers every failure listed here. + """ + process = self._started() + call_id = str(next(self._call_ids)) + process.stdin.write(encode_invoke(call_id=call_id, name=name, args=args)) + process.stdin.flush() + + line = process.stdout.readline() + if not line: + raise RuntimeError( + f"the worker child for {self._entrypoint!r} closed the pipe " + f"while answering {name!r}" + ) + frame = decode(line) + if frame["type"] == "result": + return frame["value"] + if frame["type"] == "error": + raise RuntimeError(frame["error"]) + raise RuntimeError( + f"the worker child answered {name!r} with a {frame['type']!r} " + f"frame; duplex v1 answers an invoke with 'result' or 'error'" + ) + + def shutdown(self) -> None: + """Ask the child to exit, then make sure it did. + + Idempotent: a second call is a no-op, so the explicit abort path and + the server's teardown can both call it without racing to reap the same + process twice. + + The child is *asked* first (a ``shutdown`` frame, then EOF on its + stdin) so an in-flight call can finish; only a child that ignores both + is terminated. + """ + process = self._process + if process is None or self._stopped: + return + self._stopped = True + + self._tell(process, encode_shutdown()) + self._close(process.stdin) + try: + process.wait(timeout=_EXIT_TIMEOUT) + except subprocess.TimeoutExpired: + process.terminate() + try: + process.wait(timeout=_TERMINATE_TIMEOUT) + except subprocess.TimeoutExpired: + # Named rather than waited on: teardown must finish, and a + # child this deaf is a bug report, not a thing to block on. + logger.warning( + "worker child for %r ignored terminate; giving up on it", + self._entrypoint, + ) + self._close(process.stdout) + + def _started(self) -> _ChildProcess: + """The running child. + + Returns: + The process :meth:`start` spawned. + + Raises: + RuntimeError: Nothing has been started, or it has already been + reaped. Either way there is no one to talk to, and saying so + beats an AttributeError on a ``None`` pipe. + """ + if self._process is None: + raise RuntimeError( + f"the worker child for {self._entrypoint!r} has not been " + f"started; call start() first" + ) + if self._stopped: + raise RuntimeError( + f"the worker child for {self._entrypoint!r} has been shut down" + ) + return self._process + + def _tell(self, process: _ChildProcess, line: str) -> None: + """Write one frame to the child, tolerating a pipe that is already gone. + + Args: + process: The child being told. + line: One complete NDJSON line. + """ + try: + process.stdin.write(line) + process.stdin.flush() + except (OSError, ValueError): + # A child that already exited took its pipe with it. That is the + # outcome this frame was asking for, so it is not a failure. + logger.debug( + "worker child for %r closed its pipe before shutdown was sent", + self._entrypoint, + ) + + def _close(self, stream: _ChildStdin | _ChildStdout) -> None: + """Close one end of the pipe, tolerating one that is already closed. + + Args: + stream: The stream to close. + """ + try: + stream.close() + except (OSError, ValueError): + logger.debug( + "worker child for %r had already closed a stream", + self._entrypoint, + ) diff --git a/src/molmcp/provider_worker/worker.py b/src/molmcp/provider_worker/worker.py new file mode 100644 index 0000000..1de5482 --- /dev/null +++ b/src/molmcp/provider_worker/worker.py @@ -0,0 +1,206 @@ +"""WorkerProvider — a plane served from a checkout, in a process of its own. + +This is a :class:`~molmcp.provider.Provider`: a ``name`` and a ``register``, +nothing the protocol does not already have. What is unusual is where the tools +come from. Instead of importing the plane, ``register`` starts a child process +for it, reads the catalog it greets with, and publishes proxies. The plane's +code — which may be an arbitrary checkout — never enters this interpreter, so a +plane that fails to import, or imports something incompatible, costs a child +process rather than the server. + +Teardown belongs to the server, not to whoever built the adapter. A *lifespan* +is the async context manager a server runs around its whole serving life: +everything before its ``yield`` is startup, everything after is shutdown. Once +``register`` has succeeded, this adapter wraps that context manager, so leaving +the server's lifespan reaps the child in the same place every other server +resource is released. The callable being wrapped is the one a caller passes as +``FastMCP(lifespan=...)``; FastMCP 4 keeps it in the private ``_lifespan`` +attribute and enters it inside ``_lifespan_manager``, so ``_lifespan`` is the +attribute this adapter reads and replaces. + +A ``FastMCP`` instance *does* also have a public ``lifespan``, inherited from +FastMCP's ``AggregateProvider``: an async context manager that takes no server +argument and combines the lifespans of the providers mounted on that server. +It is a different object with a different signature, and this adapter never +reads it — wrapping it would hang one plane's teardown off the aggregation of +every mounted plane. + +Two smaller exits back that lifespan up, and neither replaces it. +:meth:`shutdown` is the *explicit abort*, for the failure path and for a caller +who is done with a plane before the server is. A ``weakref.finalize`` is the last +resort for a server that is dropped without its lifespan ever being entered. +There is no separate ``atexit`` hook: a ``weakref.finalize`` already runs at +interpreter exit as well as on collection, so a second hook would only add a +second reaper to reason about, and :meth:`shutdown` being idempotent means +whichever of them fires first is the only one that does any work. + +Failure before ``register`` returns is the adapter's own to clean up: a +``register`` that raises leaves no child behind and leaves the server's +``_lifespan`` exactly as it found it, because a server that never gained this +plane must not owe it a teardown. +""" + +from __future__ import annotations + +import weakref +from contextlib import asynccontextmanager +from pathlib import Path +from typing import TYPE_CHECKING, Any + +from molmcp.provider import PROVIDER_NAME_PATTERN + +from .proxy import bind_tools +from .supervisor import Supervisor + +if TYPE_CHECKING: + from collections.abc import AsyncIterator + + from fastmcp import FastMCP + + +class WorkerProvider: + """One provider plane, loaded from a checkout in a child process. + + Args: + name: Plane id this adapter answers to, and the name the child must + greet with. Must satisfy the same pattern every other plane does. + entrypoint: The provider class the child constructs, as + ``package.module:ClassName``. + path: Directory that class is imported from — the checkout root. + + Attributes: + name: Plane id and MCP server name. Tools still register bare; a + composed core adds the namespace when it mounts the plane. + + Raises: + ValueError: *name* is not a valid plane id. A plane id becomes a + server name and a tool prefix, so it is checked where it is given + rather than where a client finally trips over it. + """ + + def __init__(self, *, name: str, entrypoint: str, path: str | Path) -> None: + if PROVIDER_NAME_PATTERN.match(name) is None: + raise ValueError( + f"{name!r} is not a valid plane id; expected a match for " + f"{PROVIDER_NAME_PATTERN.pattern}" + ) + self.name = name + self._entrypoint = entrypoint + self._path = path + self._supervisor: Supervisor | None = None + + def probe(self) -> bool: + """Whether the checkout this plane is served from is present. + + Availability is a question about the filesystem, so it is answered + from the filesystem: no child is started, because catalogs and client + configs ask this of every plane and must not pay a process each time. + + Returns: + True when ``path`` is a directory. + """ + return Path(self._path).is_dir() + + def register(self, mcp: FastMCP) -> None: + """Start the child, publish its tools, and take over teardown. + + The order is the contract. Nothing is published until the child has + greeted as the plane this adapter was built for, and the server's + lifespan is not touched until publishing has succeeded — so a failure + anywhere before that leaves no child running and leaves the server + owing this adapter no teardown. + + Args: + mcp: FastMCP server for this plane. + + Raises: + RuntimeError: The checkout is not there. A missing checkout is a + missing *directory*, not a missing wheel, so the message names + the path and the entrypoint and sends nobody to a package + index. Also raised when the child does start but the handshake + does not hold up: an unreadable first line, a protocol version + this side does not speak, an opening frame that is not a + ``hello``, or a catalog entry the proxy cannot rebuild a + signature from (a ``ProtocolError``, which is itself a + ``RuntimeError``). + ValueError: The child greeted as a different plane. Publishing its + tools here would attach one plane's tools to another's server, + under a namespace that then lies about where they came from. + """ + if not self.probe(): + raise RuntimeError( + f"the {self.name!r} plane is served from a checkout that is " + f"not there: {self._path} (entrypoint {self._entrypoint!r}). " + f"Point path= at the directory that module is imported from." + ) + + supervisor = Supervisor(entrypoint=self._entrypoint, path=self._path) + self._supervisor = supervisor + try: + hello = supervisor.start() + greeting = hello["name"] + if greeting != self.name: + raise ValueError( + f"the child for {self._entrypoint!r} greeted as the " + f"{greeting!r} plane, but this adapter serves " + f"{self.name!r}" + ) + bind_tools(mcp, hello, supervisor.invoke) + except BaseException: + # The lifespan swap has not happened, so the server owes this + # adapter no teardown, and the child is the only live resource + # the attempt created. Tool names bound before a mid-publish + # failure do stay on the server, but they proxy to a child that + # is reaped here, so calling one raises rather than hanging. + self.shutdown() + raise + + # The callable a caller passed as ``FastMCP(lifespan=...)``. FastMCP 4 + # always sets ``_lifespan`` — a server built without one gets + # ``fastmcp.server.server.default_lifespan`` — so in practice this is + # never None. The ``getattr`` default and the None branch below are + # defensive: a stub server, or a FastMCP that stopped setting the + # attribute, degrades to "still reap the child" rather than to an + # AttributeError raised out of register(). + previous = getattr(mcp, "_lifespan", None) + + @asynccontextmanager + async def wrapped(server: FastMCP) -> AsyncIterator[Any]: + """Run the server's own lifespan, then reap this plane's child. + + Args: + server: The FastMCP server being started. + + Yields: + Whatever the previous lifespan yielded — that value is the + server's application state, and swallowing it would silently + take it away from every other user of the server. + """ + if previous is None: + try: + yield {} + finally: + self.shutdown() + else: + async with previous(server) as value: + try: + yield value + finally: + self.shutdown() + + mcp._lifespan = wrapped + weakref.finalize(mcp, self.shutdown) + + def shutdown(self) -> None: + """Abort this plane's child now, without waiting for the lifespan. + + This is the explicit abort — the failure path, and a caller done with + a plane early. It is not how a registered plane is normally torn down; + that is the lifespan this adapter wrapped in :meth:`register`. + + Idempotent, and safe before ``register`` has ever run. + """ + supervisor = self._supervisor + if supervisor is None: + return + supervisor.shutdown() diff --git a/src/molmcp/providers/annotations.py b/src/molmcp/providers/annotations.py index 0e28c54..776524a 100644 --- a/src/molmcp/providers/annotations.py +++ b/src/molmcp/providers/annotations.py @@ -1,83 +1,22 @@ -"""The annotation vocabulary every plane shares. - -``ToolAnnotations`` is how a client decides whether to run a tool without -asking, so the values are a contract rather than decoration. Each provider -used to hand-roll its own set inside ``register()``, and they disagreed: -molexp's read-only tools omitted ``open_world_hint`` entirely, which reads -as *unknown* rather than *local*. - -Six constants cover every first-party tool. A provider that needs a seventh -should add it here, with the reason, rather than build one inline — the -whole point is that a named vocabulary cannot drift the way inline literals -did. +"""Re-export of the six annotation constants from the public SDK. + +The implementation lives in :mod:`molmcp.provider_sdk`; this module exists +so existing ``molmcp.providers.annotations`` imports keep working and +resolve to :data:`~molmcp.provider_sdk.READ_ONLY`, +:data:`~molmcp.provider_sdk.READ_REMOTE`, +:data:`~molmcp.provider_sdk.MUTATION`, +:data:`~molmcp.provider_sdk.LOCAL_MUTATION`, +:data:`~molmcp.provider_sdk.APPEND_WRITE`, and +:data:`~molmcp.provider_sdk.IDEMPOTENT_WRITE`. """ -from __future__ import annotations - -from mcp.types import ToolAnnotations - -#: Reads local state and nothing else. Safe to call, safe to repeat. -READ_ONLY = ToolAnnotations( - read_only_hint=True, - destructive_hint=False, - idempotent_hint=True, - open_world_hint=False, -) - -#: Reads, but reaches a scheduler, a browser, or the network to do it. Still -#: safe to call; the answer can change underneath you. -READ_REMOTE = ToolAnnotations( - read_only_hint=True, - destructive_hint=False, - idempotent_hint=False, - open_world_hint=True, -) - -#: Changes state beyond this machine and cannot be trivially undone — -#: submitting to a cluster, cancelling a remote job, driving a browser. A -#: client should confirm before calling one of these. -MUTATION = ToolAnnotations( - read_only_hint=False, - destructive_hint=True, - idempotent_hint=False, - open_world_hint=True, -) - -#: Rewrites or removes local state, and resumes rather than duplicating when -#: called again. -#: -#: Destructiveness and reach are independent axes, and the first cut of this -#: vocabulary fused them: every destructive tool had to claim it touched an -#: open world. molexp's ``run_adoption`` is the case that exposed it — move -#: mode unlinks source files, it resumes from a ledger, and it never leaves -#: the filesystem. Forcing it onto MUTATION would have made it lie twice. -LOCAL_MUTATION = ToolAnnotations( - read_only_hint=False, - destructive_hint=True, - idempotent_hint=True, - open_world_hint=False, -) - -#: Adds to a local record; calling it twice adds twice. -#: -#: Additive, so *not* destructive — the spec defines the two as opposites. -#: What a caller needs to know is that a retry is not free, which is what -#: ``idempotent_hint=False`` says. Flagging it destructive instead would make -#: a client confirm every append, which is noise. -APPEND_WRITE = ToolAnnotations( - read_only_hint=False, - destructive_hint=False, - idempotent_hint=False, - open_world_hint=False, -) - -#: Create-or-get. Writes, but calling it twice leaves the same state, so it -#: is not a destructive surface even though it is not a read. -IDEMPOTENT_WRITE = ToolAnnotations( - read_only_hint=False, - destructive_hint=False, - idempotent_hint=True, - open_world_hint=False, +from molmcp.provider_sdk import ( + APPEND_WRITE, + IDEMPOTENT_WRITE, + LOCAL_MUTATION, + MUTATION, + READ_ONLY, + READ_REMOTE, ) __all__ = [ diff --git a/src/molmcp/providers/base.py b/src/molmcp/providers/base.py index 44c1795..d2b32b6 100644 --- a/src/molmcp/providers/base.py +++ b/src/molmcp/providers/base.py @@ -1,176 +1,12 @@ -"""The shape a provider plane shares: declare tools, let the base register them. +"""Re-export of ProviderBase, ToolSpec, and tool from the public SDK. -Each provider used to spend most of its class on one ``register()`` method — -349, 402 and 191 lines — holding every tool as a nested function, plus its -own copy of the availability probe, the missing-package guard, and a set of -hand-rolled annotations. The duplication drifted: three probes with three -signatures, three guard messages (one of which never said how to install -anything), and annotation values that disagreed between planes. - -Here a tool is a method carrying a :func:`tool` declaration. The base -collects them, checks the upstream package once, and registers. Providers -are left holding only what is theirs: what the tools do. - -Nothing here imports a science package. ``probe`` asks the import system -whether one *could* be imported, which is what a catalog listing needs — -importing it to find out would drag a whole scientific stack into a process -that only wanted to print a list. +The implementation lives in :mod:`molmcp.provider_sdk`; this module exists +so existing ``molmcp.providers.base`` imports keep working and resolve to +:class:`~molmcp.provider_sdk.ProviderBase`, +:class:`~molmcp.provider_sdk.ToolSpec`, and +:func:`~molmcp.provider_sdk.tool`. """ -from __future__ import annotations - -import importlib.util -from collections.abc import Callable, Iterator -from dataclasses import dataclass -from typing import TYPE_CHECKING, Any, ClassVar - -from mcp.types import ToolAnnotations - -if TYPE_CHECKING: - from fastmcp import FastMCP - -#: Attribute a declared tool carries. Private by convention; read only here. -_MARKER = "__molmcp_tool__" - - -@dataclass(frozen=True, slots=True) -class ToolSpec: - """One tool a provider offers. - - Attributes: - name: The wire name a client sees after the server prefix. Bare by - contract — the plane id is already the server name. - annotations: From :mod:`molmcp.providers.annotations`. - attribute: Name of the method implementing it. - """ - - name: str - annotations: ToolAnnotations - attribute: str - - -def tool( - annotations: ToolAnnotations, *, name: str | None = None -) -> Callable[[Callable[..., Any]], Callable[..., Any]]: - """Declare a method as one of this plane's MCP tools. - - Args: - annotations: What a client needs to decide whether to confirm first. - Use a constant from :mod:`molmcp.providers.annotations`. - name: Wire name, when the method cannot carry it. ``open`` shadows a - builtin and ``exec`` is a keyword, so both are declared this way. - Defaults to the method name. - """ - - def declare(fn: Callable[..., Any]) -> Callable[..., Any]: - setattr(fn, _MARKER, (name or fn.__name__, annotations)) - return fn - - return declare - - -class ProviderBase: - """Base for a plane's provider. - - Subclasses set :attr:`name`, optionally :attr:`upstream` / - :attr:`import_name`, and declare tools with :func:`tool`. - - Attributes: - name: Plane id and MCP server name. Clients see ``__``. - upstream: Distribution to install when the plane is unavailable, as - it would be typed after ``pip install``. ``None`` means the plane - needs nothing beyond molmcp. - import_name: Module :meth:`probe` looks for. Defaults to *upstream* - with dashes folded, which is wrong often enough to be worth - setting explicitly. - """ - - name: ClassVar[str] - upstream: ClassVar[str | None] = None - import_name: ClassVar[str | None] = None - - # -- availability ------------------------------------------------- - - def probe(self) -> bool: - """Whether this plane can be served here. - - A plane whose science package is missing is a normal state, not an - error: catalogs and generated client configs omit it silently. Only - an explicit ``molmcp serve `` fails, and then loudly. - - Override when availability is not just "the package is present" — - molvis is available whenever a stage factory has been injected, - browser or no browser. - """ - module = self.import_name or ( - self.upstream.replace("-", "_") if self.upstream else None - ) - if module is None: - return True - try: - return importlib.util.find_spec(module) is not None - except (ImportError, ValueError): - # A package present but broken is not one we can serve. - return False - - def require_upstream(self) -> None: - """Raise unless this plane's package is installed. - - Raises: - RuntimeError: naming the distribution and how to install it. - """ - if self.probe(): - return - target = self.upstream or self.name - raise RuntimeError( - f"the {self.name!r} plane requires the {target!r} package. " - f"Install with: pip install {target}" - ) - - # -- registration -------------------------------------------------- - - def tool_specs(self) -> Iterator[ToolSpec]: - """Every declared tool, base classes first, in declaration order.""" - found: dict[str, ToolSpec] = {} - for klass in reversed(type(self).__mro__): - for attribute, value in vars(klass).items(): - marker = getattr(value, _MARKER, None) - if marker is None: - continue - wire_name, annotations = marker - found[attribute] = ToolSpec( - name=wire_name, annotations=annotations, attribute=attribute - ) - return iter(found.values()) - - def register(self, mcp: FastMCP) -> None: - """Attach this plane's tools to its server. - - Bound methods are handed to FastMCP directly: ``self`` is already - applied, so it never reaches the tool schema, and the docstring the - agent reads is the one on the method. - - Raises: - RuntimeError: the upstream package is missing. - ValueError: two methods claim the same wire name. Overriding by - *attribute* is intended — a subclass redefining a tool - replaces it — but two distinct methods claiming one name is - one tool shadowing another, and which survives would depend - on MRO order. - """ - self.require_upstream() - claimed: dict[str, str] = {} - for spec in self.tool_specs(): - previous = claimed.get(spec.name) - if previous is not None: - raise ValueError( - f"{type(self).__name__} declares the tool name " - f"{spec.name!r} twice: {previous}() and {spec.attribute}()" - ) - claimed[spec.name] = spec.attribute - mcp.tool(name=spec.name, annotations=spec.annotations)( - getattr(self, spec.attribute) - ) - +from molmcp.provider_sdk import ProviderBase, ToolSpec, tool __all__ = ["ProviderBase", "ToolSpec", "tool"] diff --git a/src/molmcp/providers/molexp/adopt/runner.py b/src/molmcp/providers/molexp/adopt/runner.py index 3a4c9dd..f39cefb 100644 --- a/src/molmcp/providers/molexp/adopt/runner.py +++ b/src/molmcp/providers/molexp/adopt/runner.py @@ -357,7 +357,8 @@ def _entries( """Ledger entries in deterministic order: run node, its files, its ingest. The ingest entry is planned up front so a resumed adoption can see it is - already done — ``metrics.jsonl`` is append-only, and re-ingesting a run + already done — the metrics WAL is append-only (densified to Zarr on flush), + and re-ingesting a run would double every curve in it. """ entries: list[Entry] = [] diff --git a/src/molmcp/providers/molexp/adopt/survey.py b/src/molmcp/providers/molexp/adopt/survey.py index 7bc244c..669d7c7 100644 --- a/src/molmcp/providers/molexp/adopt/survey.py +++ b/src/molmcp/providers/molexp/adopt/survey.py @@ -40,6 +40,16 @@ } ) + +def _has_metrics_surface(run_dir: Path) -> bool: + """True when a run already has metrics (JSONL WAL and/or dense Zarr).""" + metrics = run_dir / "metrics" + if (metrics / "metrics.jsonl").is_file(): + return True + zarr_marker = metrics / "zarr" / "zarr.json" + return zarr_marker.is_file() + + #: Suffixes that make a file read as the output of a simulation run. _ARTIFACT_SUFFIXES: frozenset[str] = frozenset( { @@ -304,9 +314,7 @@ def survey_source( files=files[rel], subdirs=children[rel], logs=tuple(detect(node_path(base, rel))) if kinds[rel] == RUN else (), - has_metrics_buffer=( - node_path(base, rel) / "metrics" / "metrics.jsonl" - ).is_file(), + has_metrics_buffer=_has_metrics_surface(node_path(base, rel)), ) for rel in sorted(children) ) diff --git a/src/molmcp/providers/molexp/layout.py b/src/molmcp/providers/molexp/layout.py index 380550c..92764da 100644 --- a/src/molmcp/providers/molexp/layout.py +++ b/src/molmcp/providers/molexp/layout.py @@ -37,7 +37,7 @@ class LayoutLevel: container="", dir_template=".", entity_file="workspace.json", - children_index_file="project.json", + children_index_file="projects.json", child_kind="project", id_rule="workspace root directory; no id in the path", ), @@ -47,7 +47,7 @@ class LayoutLevel: container="projects", dir_template="projects/", entity_file="project.json", - children_index_file="experiment.json", + children_index_file="experiments.json", child_kind="experiment", id_rule="slug(name), kebab-case, no prefix", ), @@ -57,7 +57,7 @@ class LayoutLevel: container="experiments", dir_template="projects//experiments/", entity_file="experiment.json", - children_index_file="run.json", + children_index_file="runs.json", child_kind="run", id_rule="slug(name) or explicit id, kebab-case, no prefix", ), @@ -79,8 +79,9 @@ class LayoutLevel: "Container subdir is the child kind pluralized: projects/, experiments/, runs/.", "Project/Experiment dir names are slugified ids with no prefix.", "Run dirs are always prefixed run- under runs/.", - "Entity metadata filename is the level's class name snake_case + .json.", - "Children-index filename in a parent is the *child* class name snake_case + .json.", + "Entity metadata filename is singular (project.json / experiment.json / run.json).", + "Children-index filename on the parent is plural " + "(projects.json / experiments.json / runs.json).", "Every concept dir has meta.yaml with a registered type.", "Run hot state lives in _ops/run.json (not in the run.json entity file).", ) @@ -95,16 +96,16 @@ def render_tree() -> str: return ( "workspace_root/\n" "├── workspace.json\n" - "├── project.json # children INDEX of projects (derived)\n" + "├── projects.json # children INDEX of projects (derived, plural)\n" "├── meta.yaml\n" "└── projects//\n" - " ├── project.json\n" - " ├── experiment.json # children INDEX\n" + " ├── project.json # entity (singular)\n" + " ├── experiments.json # children INDEX (plural)\n" " └── experiments//\n" - " ├── experiment.json\n" - " ├── run.json # children INDEX\n" + " ├── experiment.json # entity (singular)\n" + " ├── runs.json # children INDEX (plural)\n" " └── runs/run-/\n" - " ├── run.json\n" + " ├── run.json # entity (singular)\n" " ├── meta.yaml\n" " └── _ops/run.json\n" ) @@ -148,13 +149,16 @@ def child_dirs(parent: Path) -> list[Path]: ) -def validate_workspace(root: Path) -> dict[str, Any]: +def validate_workspace(root: Path | str) -> dict[str, Any]: """Lint *root* via molexp's layout checker; return the agent-facing report. Thin wrapper around :func:`molexp.workspace.validate_workspace`. The MCP tool of the same name (``validate_workspace``) returns this dict so an agent can see which errors need fixing. + + *root* may be a local path or a host-qualified serve label + (``Arrhenius:/home/…``). """ - from molexp.workspace import validate_workspace as _molexp_validate + from .resolve import validate_workspace_report - return _molexp_validate(Path(root)).to_dict() + return validate_workspace_report(root) diff --git a/src/molmcp/providers/molexp/provider.py b/src/molmcp/providers/molexp/provider.py index d6f250b..56fab5b 100644 --- a/src/molmcp/providers/molexp/provider.py +++ b/src/molmcp/providers/molexp/provider.py @@ -64,25 +64,29 @@ def _csv_mapping(step_column: str | None, series_columns: list[str] | None): def _open_workspace(path: str | Path): - from molexp.workspace import Workspace + """Open local or host-qualified (``Host:/abs``) workspace.""" + from .resolve import open_workspace - return Workspace(Path(path).expanduser().resolve()) + return open_workspace(path) def _resolve_workspace(workspace: str | None = None): from molexp.workspace import Workspace + from .resolve import open_workspace + if workspace: - return Workspace(Path(workspace).expanduser().resolve()) + return open_workspace(workspace) configured = _configured_workspace() if configured: - return Workspace(Path(configured).expanduser().resolve()) + return open_workspace(configured) cwd = Path.cwd() if (cwd / "workspace.json").is_file() or (cwd / "meta.yaml").is_file(): return Workspace(cwd) raise RuntimeError( - "MolexpProvider could not resolve a workspace. Pass workspace= path, " - "run `molmcp config set molexp.workspace `, or run from a " + "MolexpProvider could not resolve a workspace. Pass workspace= path " + "(local or host-qualified like Arrhenius:/home/…), run " + "`molmcp config set molexp.workspace `, or run from a " "directory containing workspace.json." ) @@ -95,7 +99,15 @@ class MolexpProvider(ProviderBase): import_name = "molexp" def __init__(self, workspace: str | Path | None = None) -> None: - self._workspace = Path(workspace).expanduser().resolve() if workspace else None + # Keep host-qualified labels as strings (Path would mangle Host:/abs). + from .resolve import is_host_qualified + + if workspace is None: + self._workspace: str | Path | None = None + elif is_host_qualified(str(workspace)): + self._workspace = str(workspace).strip() + else: + self._workspace = Path(workspace).expanduser().resolve() # -- workspace resolution ------------------------------------------- @@ -161,7 +173,8 @@ def list_experiments( from .scaffold import list_experiments as _list_experiments ws = self._get_workspace(workspace) - return _list_experiments(ws.resolve(), project_id) + # Pass the live Workspace so remote host-qualified roots keep their FS. + return _list_experiments(ws, project_id) @tool(READ_ONLY) def list_runs( @@ -237,17 +250,14 @@ def validate_workspace(self, path: str) -> dict[str, Any]: * ``next_actions`` — deduplicated remediations, errors first. Do not invent a layout by hand; fix what this report lists. + + *path* may be a local absolute path or a host-qualified serve label + (``Arrhenius:/home/…`` / ``user@host:/data``) — same forms as + ``molexp validate -ws``. """ - from molexp.workspace import validate_workspace as _validate + from .resolve import validate_workspace_report - root = Path(path).expanduser().resolve() - report = _validate(root) - payload = report.to_dict() - payload["path"] = payload.get("root", str(root)) - payload["is_workspace"] = (root / "workspace.json").is_file() or ( - root / "meta.yaml" - ).is_file() - return payload + return validate_workspace_report(path) # -- scaffold (create-or-get) ---------------------------------------- @@ -275,12 +285,13 @@ def add_project( """Create-or-get a project under the workspace (idempotent on slug). Prefer this (or omit workspace to use MOLEXP_WORKSPACE) when the user - asks to create a project. + asks to create a project. ``workspace`` may be local or host-qualified + (``Arrhenius:/home/…``). """ from .scaffold import add_project as _add_project ws = self._get_workspace(workspace) - return self._scaffold_result(_add_project, ws.resolve(), name) + return self._scaffold_result(_add_project, ws, name) @tool(IDEMPOTENT_WRITE) def add_experiment( @@ -293,7 +304,7 @@ def add_experiment( from .scaffold import add_experiment as _add_experiment ws = self._get_workspace(workspace) - return self._scaffold_result(_add_experiment, ws.resolve(), project_id, name) + return self._scaffold_result(_add_experiment, ws, project_id, name) @tool(IDEMPOTENT_WRITE) def create_run( @@ -308,7 +319,7 @@ def create_run( ws = self._get_workspace(workspace) return self._scaffold_result( - _create_run, ws.resolve(), project_id, experiment_id, params=params + _create_run, ws, project_id, experiment_id, params=params ) # -- adoption: legacy data directory → four-tier workspace ----------- @@ -483,8 +494,9 @@ def ingest_metrics( ) -> dict[str, Any]: """Convert a run's foreign logs into its host metrics buffer. - Additive and **not idempotent**: ``metrics/metrics.jsonl`` is - append-only, so ingesting the same run twice doubles its curves. + Additive and **not idempotent**: the metrics WAL is append-only and + densified into ``metrics/zarr/`` on flush — ingesting the same run + twice doubles its curves. Undo by deleting ``/metrics/``. Source logs are never deleted, rewritten, moved, or truncated. diff --git a/src/molmcp/providers/molexp/resolve.py b/src/molmcp/providers/molexp/resolve.py new file mode 100644 index 0000000..79b6fcb --- /dev/null +++ b/src/molmcp/providers/molexp/resolve.py @@ -0,0 +1,107 @@ +"""Open a molexp Workspace from a local path or host-qualified serve label. + +Local agents call MCP tools with either: + +* an absolute local path (``/Users/me/ws``), or +* a serve-style remote label (``Arrhenius:/home/…``, ``user@host:/data``) + +Both resolve through molexp's single target stack so navigation / scaffold +share the same SSH filesystem as ``molexp validate -ws Host:/path``. +""" + +from __future__ import annotations + +import re +from pathlib import Path +from typing import TYPE_CHECKING, Any + +if TYPE_CHECKING: + from molexp.workspace import Workspace + +#: SCP / serve form: optional ``user@``, host, absolute remote path after ``:``. +_HOST_QUALIFIED_RE = re.compile(r"^(?:[a-zA-Z0-9_.-]+@)?[a-zA-Z0-9_.-]+:(/|~).+$") + + +def is_host_qualified(spec: str) -> bool: + """True when *spec* is ``Host:/abs`` / ``user@host:/abs`` (not a URL).""" + s = (spec or "").strip() + if not s or "://" in s: + return False + return bool(_HOST_QUALIFIED_RE.match(s)) + + +def open_workspace(spec: str | Path) -> Workspace: + """Open a :class:`~molexp.workspace.Workspace` for *spec* (local or remote). + + Host-qualified labels use molexp's ``resolve_target`` + remote + ``FileSystem`` so all Folder I/O goes over SSH. Local paths use the + default local filesystem. + """ + from molexp.workspace import Workspace + from molexp.workspace.target import resolve_target, target_to_filesystem + + raw = str(spec).strip() + if not raw: + raise ValueError("workspace spec is empty") + + if is_host_qualified(raw): + target, _transport = resolve_target(raw) + fs = target_to_filesystem(target) + root = str(target.path) + return Workspace(root, fs=fs) + + root = Path(raw).expanduser().resolve() + return Workspace(root) + + +def workspace_spec_string(workspace: str | Path | Workspace) -> str: + """Canonical string form of a workspace argument for re-open / display.""" + if isinstance(workspace, (str, Path)): + return str(workspace).strip() + resolve = getattr(workspace, "resolve", None) + if callable(resolve): + return str(resolve()) + return str(workspace) + + +def as_workspace(workspace: str | Path | Workspace) -> Workspace: + """Coerce a path/spec or live Workspace into a Workspace.""" + from molexp.workspace import Workspace as Ws + + if isinstance(workspace, Ws): + return workspace + return open_workspace(workspace) + + +def validate_workspace_report(spec: str | Path) -> dict[str, Any]: + """Lint *spec* (local path or host-qualified) via molexp validate. + + Returns the agent-facing report dict (same shape as + ``molexp validate --json`` / MCP ``validate_workspace``). + """ + from molexp.workspace import validate_workspace as _validate + + raw = str(spec).strip() + if is_host_qualified(raw): + ws = open_workspace(raw) + root = str(ws.resolve()) + report = _validate(root, fs=ws._fs) + payload = report.to_dict() + payload["path"] = raw + payload["root"] = root + payload["remote"] = True + # Marker check through the remote fs (not local Path). + has_json = ws._fs.exists(ws._fs.join(root, "workspace.json")) + has_yaml = ws._fs.exists(ws._fs.join(root, "meta.yaml")) + payload["is_workspace"] = has_json or has_yaml + return payload + + root = Path(raw).expanduser().resolve() + report = _validate(root) + payload = report.to_dict() + payload["path"] = payload.get("root", str(root)) + payload["is_workspace"] = (root / "workspace.json").is_file() or ( + root / "meta.yaml" + ).is_file() + payload["remote"] = False + return payload diff --git a/src/molmcp/providers/molexp/scaffold.py b/src/molmcp/providers/molexp/scaffold.py index 5141f60..8e7e8c1 100644 --- a/src/molmcp/providers/molexp/scaffold.py +++ b/src/molmcp/providers/molexp/scaffold.py @@ -2,6 +2,9 @@ All mutations are create-or-get via molexp public API. Never executes runs, sweeps, or science workflows. + +Workspace arguments accept a local path, a host-qualified serve label +(``Arrhenius:/home/…``), or a live :class:`~molexp.workspace.Workspace`. """ from __future__ import annotations @@ -9,9 +12,14 @@ from pathlib import Path from typing import Any +from .resolve import as_workspace, is_host_qualified, open_workspace + #: Settings key holding the default workspace path. _WORKSPACE_SETTING = "molexp.workspace" +#: Accept path string, Path, or already-open Workspace. +WorkspaceArg = str | Path | Any + def _configured_workspace() -> str: """Default workspace from settings, or empty when unset.""" @@ -29,12 +37,31 @@ def materialize_workspace( at by ``MOLEXP_WORKSPACE`` (or under a path that already has a parent ``workspace.json``). "Create a project" is :func:`add_project`, not a nested workspace. + + Host-qualified remote paths open via SSH and call ``materialize()`` + on the remote filesystem (no local mkdir). """ from molexp.workspace import Workspace - root = Path(path).expanduser().resolve() + raw = str(path).strip() + if is_host_qualified(raw): + ws = open_workspace(raw) + if name and name != "workspace": + # Name is fixed at construct time for local; remote open reuses root. + pass + ws.materialize() + return { + "path": raw, + "root": str(ws.resolve()), + "name": ws.name, + "id": getattr(ws, "id", ws.name), + "materialized": True, + "remote": True, + } + + root = Path(raw).expanduser().resolve() session = _configured_workspace() - if session: + if session and not is_host_qualified(session): session_root = Path(session).expanduser().resolve() if root != session_root and _is_relative_to(root, session_root): raise RuntimeError( @@ -60,6 +87,7 @@ def materialize_workspace( "name": ws.name, "id": getattr(ws, "id", ws.name), "materialized": True, + "remote": False, } @@ -97,11 +125,9 @@ def _folder_path(folder: object) -> str: return str(path) -def add_project(workspace: str | Path, name: str) -> dict[str, Any]: +def add_project(workspace: WorkspaceArg, name: str) -> dict[str, Any]: """``ws.add_project(name)`` — idempotent on slug.""" - from molexp.workspace import Workspace - - ws = Workspace(Path(workspace).expanduser().resolve()) + ws = as_workspace(workspace) ws.materialize() project = ws.add_project(name) return { @@ -112,14 +138,12 @@ def add_project(workspace: str | Path, name: str) -> dict[str, Any]: def add_experiment( - workspace: str | Path, + workspace: WorkspaceArg, project_id: str, name: str, ) -> dict[str, Any]: """``project.add_experiment(name)`` — idempotent on slug.""" - from molexp.workspace import Workspace - - ws = Workspace(Path(workspace).expanduser().resolve()) + ws = as_workspace(workspace) project = _require_project(ws, project_id) experiment = project.add_experiment(name) return { @@ -130,11 +154,9 @@ def add_experiment( } -def list_experiments(workspace: str | Path, project_id: str) -> list[dict[str, Any]]: +def list_experiments(workspace: WorkspaceArg, project_id: str) -> list[dict[str, Any]]: """List experiments under a project (read-only).""" - from molexp.workspace import Workspace - - ws = Workspace(Path(workspace).expanduser().resolve()) + ws = as_workspace(workspace) project = _require_project(ws, project_id) return [ { @@ -147,15 +169,13 @@ def list_experiments(workspace: str | Path, project_id: str) -> list[dict[str, A def create_run( - workspace: str | Path, + workspace: WorkspaceArg, project_id: str, experiment_id: str, params: dict[str, Any] | None = None, ) -> dict[str, Any]: """Scaffold ``add_run(params=…)`` only — leaves the run pending.""" - from molexp.workspace import Workspace - - ws = Workspace(Path(workspace).expanduser().resolve()) + ws = as_workspace(workspace) project = _require_project(ws, project_id) try: experiment = project.get_experiment(experiment_id) diff --git a/src/molmcp/runtime.py b/src/molmcp/runtime.py index 2dd2c72..3b114d2 100644 --- a/src/molmcp/runtime.py +++ b/src/molmcp/runtime.py @@ -1,17 +1,147 @@ -"""Application-layer assembly of the source collection.""" +"""Application-layer assembly of the source collection. + +:func:`build_collection` turns one resolved :class:`~molmcp.config.AppConfig` +into the :class:`~molmcp.collection.CollectionIndex` the molcrafts core +searches, and is the only place discovery's capability overlays are put in +order. Two helpers keep it company. ``_session_capability_overlays`` builds +the overlays an activated harness checkout contributes, and lives here +because that concatenation is its only consumer. :func:`resolved_cache_dir` +owns the fallback for an unset ``cacheDir``, so that no module beyond this +one and the CLI has to import :mod:`molmcp.discovery` to find a cache root. +""" from __future__ import annotations +import importlib import json +import sys +from collections.abc import Sequence +from pathlib import Path from .collection import CollectionIndex, SourceBinding +from .components.models import ComponentSpec from .config import AppConfig from .discovery import DiscoveryConfig, DiscoveryEngine from .discovery.config import DEFAULT_EXCLUDES +from .discovery.overlay import CapabilityOverlay, load_overlays + + +class OverlayLoadError(ValueError): + """Raised when a harness checkout component yields no usable overlay. + + A *harness checkout* is one commit of the user's own agent-tooling + repository, unpacked on disk. Its ``harness.toml`` catalog may declare + *capability overlays* — objects that layer domain knowledge onto the code + graph discovery builds. This is what a catalog row promising one and + delivering something else raises. Being a ``ValueError``, it is already + caught by a caller that only wants "the stack could not be built". + """ + + +def _session_capability_overlays( + seeds: Sequence[ComponentSpec], base: Path +) -> tuple[CapabilityOverlay, ...]: + """Construct the activated checkout's capability overlays in this process. + + Every seed names a ``module:object`` entrypoint. The directory holding + the seed's ``path`` inside *base* goes on ``sys.path``, the module + half is imported, and the named object is called as a factory. The result + must satisfy :class:`~molmcp.discovery.overlay.CapabilityOverlay`; one + that does not is a named error rather than a skipped warning, because a + catalog row that declares an overlay and delivers something else is a + broken catalog, not a missing optional extra. + + That directory is the *parent* of the seed's path, taken unconditionally, + and it is prepended to ``sys.path`` for the rest of the process — nothing + takes it back off, so a checkout module that shadows an installed one goes + on shadowing it long after the graph is built. The provider arm resolves + its import root by a different rule (``molmcp.harness`` / ``_import_root`` + uses a path that names a directory as it stands); that function is where + the reason the two coexist is written down. + + This import runs in-process while a checkout *provider* runs in a + subprocess of its own. The difference is session state, not trust. A + provider owns a long-lived MCP session — an ``exec`` namespace and an + event journal that outlive every individual call — so it needs a process + to hold that state and to be torn down together with it. An overlay is + called once, at graph-build time, and keeps nothing between calls. The + trust model is single and covers both: the harness repository is the + user's own code, on the same footing as the kernel a notebook executes. + + Args: + seeds: Overlay ``ComponentSpec`` rows read from the harness catalog. + base: The directory this source's component paths resolve under — + :meth:`molmcp.harness.ComponentFold.root_for`'s answer. + Deliberately not named for a checkout root: it is the tree + ``harness.toml`` sits at only while that source's catalog + declares no ``component_root``, and the two part company the + moment one does. This function is not told which case it is in + and does not need to be; it takes a base directory and knows + nothing about where it came from. + + Returns: + One overlay instance per seed, in seed order. + + Raises: + OverlayLoadError: If a seed declares no entrypoint, or its factory + returns an object that is not a ``CapabilityOverlay``. The message + names the offending component. + ImportError: The entrypoint's module half is not importable from the + checkout — a mistyped module name, or a dependency the checkout + never declared. It reaches the caller as raised, unwrapped, + because the interpreter's own message says more about which import + failed than any rewrapping here could. + AttributeError: The module imports, but carries no object under the + entrypoint's second half. Whatever the factory itself raises + propagates the same way. + """ + overlays: list[CapabilityOverlay] = [] + for seed in seeds: + entrypoint = seed.entrypoint + if entrypoint is None: + raise OverlayLoadError(f"overlay component {seed.name!r} has no entrypoint") + module_name, _, attribute = entrypoint.partition(":") + import_root = str((base / seed.path).parent) + if import_root not in sys.path: + sys.path.insert(0, import_root) + instance = getattr(importlib.import_module(module_name), attribute)() + if not isinstance(instance, CapabilityOverlay): + raise OverlayLoadError( + f"overlay component {seed.name!r} entrypoint {entrypoint!r} " + f"returned {type(instance).__name__}, not a CapabilityOverlay" + ) + overlays.append(instance) + return tuple(overlays) + + +def resolved_cache_dir(config: AppConfig) -> Path: + """Return the one resolved cache root for this configuration. + + ``AppConfig.cache_dir`` is ``None`` until somebody configures ``cacheDir``, + so every caller that wants a real directory needs the same fallback to the + discovery default. That fallback lives here and nowhere else: one place + decides the default, and the harness store can never land in a different + directory than the discovery caches. + + :mod:`molmcp.harness` reads the root from this function precisely so that + it need not import :mod:`molmcp.discovery`. Discovery has exactly two + importers — this module and the CLI — and the harness wiring is not a third. + + Args: + config: Resolved application configuration. + + Returns: + The configured ``cache_dir``, or the discovery default + (:func:`molmcp.discovery.config.default_cache_dir`) when unset. + """ + return config.cache_dir or DiscoveryConfig().cache_dir def build_collection( - config: AppConfig, registry: object | None = None + config: AppConfig, + registry: object | None = None, + *, + extras: Sequence[CapabilityOverlay] = (), ) -> CollectionIndex: """Build one collection over every named source in ``config``. @@ -20,13 +150,27 @@ def build_collection( search as one more channel. molmcp ships no implementation: the capability manifest it used to carry had no producer anywhere in the ecosystem, so the shape was guesswork. The seam stays; the guess does not. + + This is the only place overlays are assembled: the entry-point overlays + come first, then ``extras``. The concatenation is always a list, empty + included, so the engine never falls back to discovering overlays itself. + + Args: + config: Resolved application configuration. + registry: Optional duck-typed extra search channel. + extras: Overlays contributed by an activated harness checkout, + appended after the entry-point overlays. Empty is a legal answer. + + Returns: + The assembled :class:`CollectionIndex`. """ discovery = DiscoveryConfig( - cache_dir=config.cache_dir or DiscoveryConfig().cache_dir, + cache_dir=resolved_cache_dir(config), excludes=tuple(dict.fromkeys((*DEFAULT_EXCLUDES, *config.excludes))), watch=config.watch, ) - engine = DiscoveryEngine(discovery) + overlays = [*load_overlays(), *extras] + engine = DiscoveryEngine(discovery, overlays=overlays) bindings = [ SourceBinding( name=name, diff --git a/src/molmcp/server.py b/src/molmcp/server.py index ee496a3..11dba0f 100644 --- a/src/molmcp/server.py +++ b/src/molmcp/server.py @@ -1,11 +1,11 @@ -"""Build one MCP plane server — never a multi-provider mega-server.""" +"""Build MCP servers — one focused FastMCP per plane, composed via mount.""" from __future__ import annotations import hmac import logging import os -from collections.abc import Iterable +from collections.abc import Iterable, Sequence from contextlib import asynccontextmanager from pathlib import Path @@ -14,7 +14,15 @@ from mcp.types import ToolAnnotations from .collection import CollectionIndex +from .components import ComponentKind from .config import AppConfig, load_config +from .harness import ( + Checkout, + activated_checkouts, + checkout_planes, + fold_components, + servable_sources, +) from .mcp_provider import MolCraftsContextProvider from .middleware import ( MissingAnnotationsError, @@ -23,13 +31,22 @@ assert_plane_tool_names, validate_tool_annotations, ) -from .planes import BUILTIN_PLANE_IDS, list_plane_infos, route_task +from .planes import ( + BUILTIN_PLANE_IDS, + CORE_PLANE_ID, + GONE_PLANE_IDS, + core_disable_message, + gone_plane_message, + list_plane_infos, + route_task, +) from .provider import ( PROVIDER_NAME_PATTERN, Provider, discover_providers, ) -from .runtime import build_collection +from .runtime import _session_capability_overlays, build_collection +from .settings import HarnessSource, load_settings logger = logging.getLogger(__name__) @@ -41,6 +58,71 @@ ) +def _create_core_plane( + *, + collection: CollectionIndex | None, + config: AppConfig | str | Path | None, + extras: Sequence[object], + enable_path_safety: bool, + enable_response_limit: bool, + response_limit_bytes: int, + validate_annotations: bool, + instructions: str | None, +) -> FastMCP: + """Assemble the ``molcrafts`` core plane and its collection lifespan. + + The core is the one plane that owns a discovery collection rather than a + product's tools, so it is also the one that needs a lifespan: the + collection is opened when the server starts and closed when it stops, and + that ``finally`` is the only place either happens. + + Args: + collection: Injected collection (tests, embedding). When ``None`` the + collection is built from *config*. + config: ``molcrafts.json`` path or :class:`~molmcp.config.AppConfig`. + extras: Session capability overlays concatenated after the entry-point + ones. Empty for a focused core plane. + enable_path_safety: Attach the path-safety middleware. + enable_response_limit: Attach the response-limit middleware. + response_limit_bytes: Ceiling that middleware enforces. + validate_annotations: Fail startup if a tool lacks ToolAnnotations. + instructions: Override the default core instructions. + + Returns: + The core :class:`FastMCP` server, tools registered and validated. + """ + app_config, coll = _resolve_collection(collection, config, extras=extras) + auth = _environment_auth(app_config) if app_config is not None else None + + @asynccontextmanager + async def lifespan(_server): + coll.start() + try: + yield {} + finally: + coll.close() + + runtime_status: dict[str, object] = { + "plane": CORE_PLANE_ID, + "transport": ( + app_config.server.transport if app_config is not None else "injected" + ), + } + mcp = _base_server( + CORE_PLANE_ID, + instructions=instructions or _molcrafts_instructions(), + auth=auth, + lifespan=lifespan, + enable_path_safety=enable_path_safety, + enable_response_limit=enable_response_limit, + response_limit_bytes=response_limit_bytes, + ) + MolCraftsContextProvider(coll, runtime_status).register(mcp) + _register_core_routing(mcp) + _validate(mcp, validate_annotations, plane_id=CORE_PLANE_ID) + return mcp + + def create_plane( plane: str, *, @@ -49,6 +131,7 @@ def create_plane( provider: Provider | None = None, providers: Iterable[Provider] | None = None, discover_entry_points: bool = True, + extras: Sequence[object] = (), enable_path_safety: bool = True, enable_response_limit: bool = True, response_limit_bytes: int = 256 * 1024, @@ -58,7 +141,7 @@ def create_plane( """Build a **single-plane** FastMCP server. Args: - plane: Plane id (``catalog``, ``molcrafts``, or a provider name such + plane: Plane id (``molcrafts`` core, or a provider name such as ``molvis``). This becomes the MCP server name clients see. collection: Injected discovery collection (tests / embedding). Required only for ``molcrafts`` when *config* is not used. @@ -67,9 +150,17 @@ def create_plane( is built from it. provider: Explicit provider instance for provider planes (tests). providers: Deprecated alias for a single-item explicit provider list; - if more than one is passed, raises — multi-provider servers are gone. + if more than one is passed, raises — use :func:`create_stack`. discover_entry_points: Load the matching ``molmcp.providers`` entry point when *provider* is not injected. + extras: Capability overlays an activated harness checkout contributed. + A *capability overlay* is an object that layers domain knowledge + onto the code graph discovery builds, after that graph is + resolved; ``molmcp.discovery.overlay`` owns the protocol. These + are appended to the entry-point overlays when this call builds the + collection, and ignored when *collection* is injected — whoever + built that collection already chose its overlays. They stay opaque + here: naming their type would import discovery into this module. enable_path_safety / enable_response_limit / response_limit_bytes: Middleware toggles. validate_annotations: Fail startup if tools lack ToolAnnotations. @@ -82,12 +173,14 @@ def create_plane( plane_id = plane.strip().lower() if not plane_id: raise ValueError("plane id must be non-empty") + if plane_id in GONE_PLANE_IDS: + raise ValueError(gone_plane_message(plane_id)) if providers is not None: explicit_list = list(providers) if len(explicit_list) > 1: raise ValueError( - "multi-provider servers are removed; serve one plane per process" + "create_plane serves one plane; compose providers with create_stack" ) if provider is not None and explicit_list: raise ValueError("pass provider= or providers=[one], not both") @@ -102,50 +195,17 @@ def create_plane( f"serve {getattr(provider, 'name', 'it')!r} as its own plane" ) - if plane_id == "catalog": - mcp = _base_server( - plane_id, - instructions=instructions or _catalog_instructions(), - auth=None, - lifespan=None, + if plane_id == CORE_PLANE_ID: + return _create_core_plane( + collection=collection, + config=config, + extras=extras, enable_path_safety=enable_path_safety, enable_response_limit=enable_response_limit, response_limit_bytes=response_limit_bytes, + validate_annotations=validate_annotations, + instructions=instructions, ) - _register_catalog(mcp) - _validate(mcp, validate_annotations, plane_id=plane_id) - return mcp - - if plane_id == "molcrafts": - app_config, coll = _resolve_collection(collection, config) - auth = _environment_auth(app_config) if app_config is not None else None - - @asynccontextmanager - async def lifespan(_server): - coll.start() - try: - yield {} - finally: - coll.close() - - runtime_status: dict[str, object] = { - "plane": plane_id, - "transport": ( - app_config.server.transport if app_config is not None else "injected" - ), - } - mcp = _base_server( - plane_id, - instructions=instructions or _molcrafts_instructions(), - auth=auth, - lifespan=lifespan, - enable_path_safety=enable_path_safety, - enable_response_limit=enable_response_limit, - response_limit_bytes=response_limit_bytes, - ) - MolCraftsContextProvider(coll, runtime_status).register(mcp) - _validate(mcp, validate_annotations, plane_id=plane_id) - return mcp # Provider plane — one product, bare tool names, server name = plane id. resolved = _resolve_provider( @@ -176,6 +236,197 @@ async def lifespan(_server): return mcp +def create_stack( + *, + collection: CollectionIndex | None = None, + config: AppConfig | str | Path | None = None, + providers: Iterable[Provider] | None = None, + disable: Iterable[str] = (), + discover_entry_points: bool = True, + enable_path_safety: bool = True, + enable_response_limit: bool = True, + response_limit_bytes: int = 256 * 1024, + validate_annotations: bool = True, + instructions: str | None = None, +) -> FastMCP: + """Build the molcrafts core and mount enabled providers (FastMCP composition). + + Provider tools are namespaced with the plane id (``molvis_open``). Core + tools stay bare (``packages``, ``open``, ``route``). ``molcrafts`` cannot + be disabled. + + This is also the only composition root the activated harness checkouts + reach — one commit per named harness source (see + :data:`molmcp.harness.SUPPORTED_CAPABILITIES`), already unpacked under + the cache directory. It has two arms, each with an owner: the *overlay* + arm builds the collection (it runs when *collection* is not injected), + the *provider* arm enumerates planes (it runs when *providers* is not + injected and entry-point discovery is on). Injecting one arm's answer + skips that arm and only that arm. Injecting both means the caller has + answered everything, so the harness sources are never even read. + + An arm that would reach for a checkout reads + :func:`~molmcp.settings.load_settings` once and validates every named + source. An empty list — no source named at all — serves exactly as this + did before the harness existed; an entry missing a coordinate is a + :class:`~molmcp.config.ConfigurationError` rather than a guess at the + missing half, and no entry is skipped in favour of the next. + + Every activated source contributes, and each arm folds them itself + (:func:`molmcp.harness.fold_components`): components are taken in the + order the ``harness`` settings list names their sources, and a component + id two sources both declare is kept once, from the earlier entry, with the + later one reported. For a provider that id *is* a plane name, so the fold + is also what keeps two sources' ``provider.demo`` from mounting twice + under one namespace — and the folded name set, not the first catalog, is + what entry-point planes are XORed against. + + Args: + collection: Injected discovery collection. Supplying one answers the + overlay arm: nothing is built here, so no checkout overlay is + loaded — whoever built that collection already chose its overlays. + config: ``molcrafts.json`` path or :class:`AppConfig`, used for the + core plane and passed on to every plane mounted under it. + providers: Explicit planes to mount. Supplying them answers the + provider arm: entry points are not enumerated and the checkout + contributes no plane. They are still filtered by *disable*. + disable: Plane ids to leave unmounted. ``molcrafts`` may not be one of + them, and a retired plane id is refused rather than ignored. + discover_entry_points: Enumerate ``molmcp.providers`` entry points + when *providers* is not injected. ``False`` with no injected + providers mounts nothing — it is not a checkout-only mode. + enable_path_safety / enable_response_limit / response_limit_bytes: + Middleware toggles, applied to every plane this builds. + validate_annotations: Fail startup if tools lack ToolAnnotations. + instructions: Override the composed core's instructions string; + mounted planes keep their own. + + Returns: + The core server, with every enabled plane already mounted on it. + + Raises: + ValueError: ``molcrafts`` was disabled, or a retired plane was named. + ConfigurationError: A named harness source cannot be served. Four + ways: an entry is missing a coordinate; an entry's ``name`` cannot + name that source's activation pointer file, because it is empty, + reserved, absolute or carries a path separator (see + :func:`molmcp.harness.pointer_path`); two entries share a name, + compared case-insensitively because both spellings resolve to one + pointer file on darwin and on Windows; or a source's activated + commit has no tree on disk. A ``ValueError`` subclass, as are + ``CatalogError`` and ``OverlayLoadError``. + CatalogError: A checkout's ``harness.toml`` failed the catalog + grammar, or asks for a capability token this runtime does not + implement — see :func:`~molmcp.components.load_harness_catalog`. + The fold raises it too, and not only about a file: a + :class:`~molmcp.harness.ComponentFold` whose checkouts and + ``component_root`` strings disagree cannot be built, and + :meth:`~molmcp.harness.ComponentFold.root_for` refuses a source + the fold was never given rather than answering with some other + source's base. Raised out of either arm. One bad catalog fails + the serve rather than being skipped in favour of its neighbours. + OverlayLoadError: A checkout overlay component's factory returned + something that is not a capability overlay — see + ``molmcp.runtime._session_capability_overlays``. + ActivationVersionError: A source's activation pointer file exists but + is not a version-1 record. Alone among these it is *not* a + ``ValueError``: a pointer this process cannot parse is not a + configuration mistake it could serve without. + """ + skipped = {str(name).strip().lower() for name in disable if str(name).strip()} + if CORE_PLANE_ID in skipped: + raise ValueError(core_disable_message()) + for name in skipped: + if name in GONE_PLANE_IDS: + raise ValueError(gone_plane_message(name)) + + build_overlays = collection is None + enumerate_planes = providers is None and discover_entry_points + plane_config: AppConfig | str | Path | None = config + checkouts: tuple[Checkout, ...] = () + if (build_overlays or enumerate_planes) and (sources := _harness_locator()): + # Resolving here rather than in activated_checkouts keeps the cache + # root the *same* already-resolved root the collection indexes under. + plane_config = _resolve_config(config) + checkouts = activated_checkouts(plane_config, sources) + + extras: tuple[object, ...] = () + if build_overlays and checkouts: + # ``_session_capability_overlays`` resolves each seed's import root + # under one base, so N checkouts is N calls concatenated in source + # order — not one call over a flattened spec list, which would resolve + # the second source's seeds under the first source's base. The base is + # the fold's answer rather than the checkout tree: a catalog may + # declare a ``component_root``, and the provider arm asks the same + # question of the same object, so neither arm can be the one that + # forgot. + overlay_fold = fold_components(checkouts, ComponentKind.OVERLAY) + extras = tuple( + overlay + for checkout in overlay_fold.checkouts + for overlay in _session_capability_overlays( + overlay_fold.specs_from(checkout.source), + overlay_fold.root_for(checkout.source), + ) + ) + + parent = create_plane( + CORE_PLANE_ID, + collection=collection, + config=plane_config, + extras=extras, + discover_entry_points=False, + enable_path_safety=enable_path_safety, + enable_response_limit=enable_response_limit, + response_limit_bytes=response_limit_bytes, + validate_annotations=validate_annotations, + instructions=instructions or _stack_instructions(), + ) + if providers is not None: + mounted: list[Provider] = [p for p in providers if p.name not in skipped] + elif not enumerate_planes: + mounted = [] + else: + provider_fold = fold_components(checkouts, ComponentKind.PROVIDER) + workers = checkout_planes(provider_fold) + # The exclusion set is an *output of the fold*, not a set built back + # out of the constructed workers: a contested ``provider.demo`` is + # kept once, so the name it claims against the entry points is claimed + # once, whichever source won it. + from_checkout = provider_fold.names + # One enumeration, and the same one this arm has always used. + # ``only_available=True`` drops a plane whose optional upstream + # package is not installed — precisely the plane a checkout is there + # to supply — and enumerating twice would construct every entry-point + # provider class a second time on every serve. + mounted = [ + p + for p in ( + *workers, + *( + plane + for plane in discover_providers(only_available=True) + if plane.name not in from_checkout + ), + ) + if p.name not in skipped + ] + + for provider in mounted: + child = create_plane( + provider.name, + provider=provider, + config=plane_config, + discover_entry_points=False, + enable_path_safety=enable_path_safety, + enable_response_limit=enable_response_limit, + response_limit_bytes=response_limit_bytes, + validate_annotations=validate_annotations, + ) + parent.mount(child, namespace=provider.name) + return parent + + def create_server( name: str | None = None, *, @@ -189,9 +440,7 @@ def create_server( """ plane_id = plane or name if plane_id is None: - raise ValueError("create_plane requires plane= (or legacy name=)") - # Strip kwargs that only applied to the mega-server. - kwargs.pop("provider_names", None) + raise ValueError("create_plane requires plane=") return create_plane(plane_id, **kwargs) @@ -216,44 +465,104 @@ def _base_server( return mcp -def _register_catalog(mcp: FastMCP) -> None: +def _register_core_routing(mcp: FastMCP) -> None: @mcp.tool(annotations=_READ_ONLY) def list_planes() -> dict[str, object]: - """List MCP planes this install can serve (connect only what you need). + """List the core connection and optional provider planes. - Each row has ``id``, ``serve_command``, ``when_to_connect``, and - ``tools_hint``. There is no mega-server — one process per plane. + Each row has ``id``, ``serve_command``, ``when_to_connect``, + ``tools_hint``, and ``disableable``. molcrafts is always on; + only provider planes can be dropped from a client config. """ planes = [p.to_dict() for p in list_plane_infos()] return { "ok": True, "planes": planes, - "model": "multi-link-on-demand", + "core": CORE_PLANE_ID, + "model": "molcrafts core + optional provider planes", "hint": ( - "Configure separate MCP server entries per plane. " - "Start with catalog + the planes route() returns." + "Default `molmcp serve` mounts enabled providers onto this " + "core (FastMCP namespace: molvis_open). " + "Drop a mount with `molmcp init --disable `." ), } @mcp.tool(annotations=_READ_ONLY) def route(task: str) -> dict[str, object]: - """Which plane(s) to connect for *task* (routing only — no science). + """Which optional provider plane(s) to connect for *task*. - Returns plane ids and ``molmcp serve `` commands. Connect those - MCP links on demand; do not invent domain MCP tools for chemistry APIs. + Routing only — no science. molcrafts is already this connection. + Do not invent domain MCP tools for chemistry APIs. """ return route_task(task) +def _resolve_config(config: AppConfig | str | Path | None) -> AppConfig: + """Accept either an already-resolved config or something to load one from. + + Resolution is this module's job and stays here. :mod:`molmcp.harness` takes + an :class:`AppConfig` already resolved, so that the harness store and its + pointers land under the very same cache root the collection indexes under + rather than under a root a second resolution might disagree about. + + Args: + config: An :class:`AppConfig`, or anything + :func:`~molmcp.config.load_config` accepts. + + Returns: + The configuration, resolved. + """ + if isinstance(config, AppConfig): + return config + return load_config(config) + + def _resolve_collection( collection: CollectionIndex | None, config: AppConfig | str | Path | None, + *, + extras: Sequence[object] = (), ) -> tuple[AppConfig | None, CollectionIndex]: if collection is not None: app_config = _resolve_config(config) if config is not None else None return app_config, collection app_config = _resolve_config(config) - return app_config, build_collection(app_config) + return app_config, build_collection(app_config, extras=extras) + + +def _harness_locator() -> tuple[HarnessSource, ...]: + """Read every named harness source, in the order the settings list them. + + Settings are read once per ``create_stack``, rooted at the working + directory the way every other caller reads them: a bare ``load_settings()`` + would hide a project's ``.molmcp/settings.json`` layer, so a source split + across the user and project files would look incomplete and be rejected. + + Reading the file is this function's whole job; which entries are servable + is :func:`~molmcp.harness.servable_sources`', so that ``molmcp harness + sync`` can apply the identical rule to the one entry it was named. The + split is what keeps the two commands from drifting: an entry this refuses + cannot be one that verb syncs. + + Returns: + Every named source in file order, each naming an origin this install + can reach — a GitHub coordinate or a checkout on disk — or the empty + tuple when no source is named, which is the un-harnessed + configuration rather than a failure. File order is carried through + :func:`~molmcp.harness.activated_checkouts` into the fold, so it is + the operator's priority control over a component two sources both + declare. Only ``name`` is read past this point: it selects that + source's activation pointer, which is where the commit to serve comes + from. The origin fields identify the repository to whatever later + fetches from it, and nothing downstream of here reads their values. + + Raises: + ConfigurationError: An entry names no origin this install can reach. + See :func:`~molmcp.harness.assert_servable` for the GitHub + locator (complete without a ref) and the local checkout that + qualify, and what each message says. + """ + return servable_sources(load_settings(Path.cwd()).harness) def _resolve_provider( @@ -292,12 +601,6 @@ def _resolve_provider( return found[plane_id] -def _resolve_config(config: AppConfig | str | Path | None) -> AppConfig: - if isinstance(config, AppConfig): - return config - return load_config(config) - - def _validate( mcp: FastMCP, validate_annotations: bool, @@ -315,27 +618,31 @@ def _validate( ) -def _catalog_instructions() -> str: +def _molcrafts_instructions() -> str: return ( - "MolCrafts MCP catalog plane — multi-link on-demand bootstrap.\n" - "1) list_planes — which product planes exist and how to serve them\n" - "2) route(task) — which plane(s) to connect for a user task\n" - "Connect only those MCP servers. Science APIs are never tools here; " - "use the molcrafts plane to discover symbols, molvis to draw, etc." + "MolCrafts knowledge core. Discover real symbols before coding.\n" + "1) list_planes — which provider mounts exist\n" + "2) route(task) — which provider namespace a task needs\n" + "3) packages — package directory; choose sources\n" + "4) outline(source, path?) — module tree\n" + "5) open(ref) — symbol page before coding\n" + "6) compose(task|refs) — budgeted multi-page pack\n" + "search/suggest are index helpers. " + "ok=false / SYMBOL_NOT_FOUND → capability gap: report the step, " + "the package/ref, and the result; do not invent the API.\n" + "knowledgeScope scopes packages/outline/open/search/compose. " + "Science APIs are never tools; invoke them in agent Python " + "or via the namespaced molvis tools." ) -def _molcrafts_instructions() -> str: +def _stack_instructions() -> str: return ( - "MolCrafts knowledge plane (OKF-style pages). " - "Codegraph is an index — do not treat scores as truth.\n" - "1) packages — package directory; choose sources\n" - "2) outline(source, path?) — module tree\n" - "3) open(ref) — symbol page before coding\n" - "4) compose(task|refs) — budgeted multi-page pack\n" - "search/suggest are index helpers. " - "ok=false / SYMBOL_NOT_FOUND → do not invent the API.\n" - "knowledgeScope scopes packages/outline/open/search/compose." + _molcrafts_instructions() + + "\nDefault serve mounts providers with FastMCP namespaces " + "(molvis_open, molq_list_jobs, molexp_list_projects). " + "`molmcp init --disable ` omits a mount. " + "If these tools are missing, tell the user to install or start molmcp." ) @@ -346,7 +653,8 @@ def _provider_instructions(plane_id: str) -> str: "Do not expect science methods as MCP tools; discover them on the " "molcrafts plane and invoke via agent Python or molvis exec.\n" f"Server name is '{plane_id}' so client tool ids look like " - f"'{plane_id}__'." + f"'{plane_id}__'. On the composed core they appear as " + f"'{plane_id}_' (FastMCP namespace)." ) @@ -380,4 +688,4 @@ def _environment_auth(config: AppConfig) -> TokenVerifier | None: return _EnvironmentTokenVerifier(environment_name) -__all__ = ["create_plane", "create_server"] +__all__ = ["create_plane", "create_server", "create_stack"] diff --git a/src/molmcp/settings.py b/src/molmcp/settings.py index 7019d95..e873f82 100644 --- a/src/molmcp/settings.py +++ b/src/molmcp/settings.py @@ -21,10 +21,14 @@ from __future__ import annotations import json -from dataclasses import dataclass, field +from collections.abc import Sequence +from dataclasses import asdict, dataclass, field, fields from pathlib import Path from typing import Any +from .components.locator import LocatorError, parse_harness_locator +from .components.models import COMPONENT_NAME_PATTERN + #: Directory name used for both the user home and a project checkout. CONFIG_DIR_NAME = ".molmcp" SETTINGS_NAME = "settings.json" @@ -49,6 +53,7 @@ class SettingsError(ValueError): "pythonEnv": str, "discoverInclude": list, "discoverExclude": list, + "harness": list, "molexp": dict, "molq": dict, } @@ -66,6 +71,135 @@ class SettingsError(ValueError): _MERGED_DICTS = ("sources", "molexp", "molq") _MERGED_LISTS = ("excludes", "knowledgeScope", "discoverInclude", "discoverExclude") +#: List-valued settings whose *elements are objects*, which the string-valued +#: editing verbs cannot author: `config set harness x` would store the list +#: ``["x"]`` and `config add harness x` would append the bare string, and both +#: write before anything validates — leaving a file every later read rejects. +#: This is a declaration, not a merge channel: ``load_settings`` never consults +#: it, so the next list-of-objects setting closes the same hole by joining this +#: tuple rather than by someone remembering to add a second branch. Joining it +#: also means shipping a ``molmcp config set`` leaf: the refusal +#: ``_reject_object_list_write`` raises derives that command from the key, so a +#: member added without its verb would hand out a command nothing resolves — +#: which a test asserts against this tuple rather than a docstring promising it. +#: The entry-key list in that same message is still ``harness``'s own, and +#: nothing fails if it goes on naming only these. +#: +#: ``harness`` is deliberately in no merge channel at all. The default branch of +#: ``load_settings`` makes the last assignment win, and ``settings_layers`` +#: yields lowest precedence first, so the most specific layer's list replaces +#: the others whole. That is the opposite of ``_MERGED_LISTS`` one line above, +#: on purpose: ``extend`` on a first-wins list would land the user file's +#: entries at the front and make the user file outrank the project file. +_OBJECT_LISTS = ("harness",) + +#: Alias given to the first harness source authored without ``--alias``. +DEFAULT_HARNESS_ALIAS = "origin" + +#: Coordinate keys the locator model retired. A file that still carries one +#: is a hard cut: re-author with ``molmcp config harness set ``. +_RETIRED_HARNESS_KEYS = frozenset({"owner", "path", "ref", "repo"}) + + +@dataclass(frozen=True, slots=True) +class HarnessSource: + """One named harness repository this install may serve components from. + + A harness is the git repository of the operator's own agent tooling — + skills, agents, rules, provider planes, discovery overlays. An install + may name several, and the order they are written in is the order they + are read in. + + The dataclass stores only what the operator wrote: an alias, a locator, + and an optional enable list. GitHub identity and the local path are + parsed from ``locator`` at construction and are properties, not fields — + they do not appear in :func:`dataclasses.asdict` or in the settings file. + Construction requires a locator; a name-only half-authored entry is not + a thing this type can represent. + + ``name`` is held to no grammar beyond "non-empty, no whitespace", + deliberately: it is user-chosen in exactly the way a ``sources`` key is, + and an operator who may name an index source ``MolCrafts`` may name a + harness source ``MolCrafts`` too. ``/`` is still refused later, when + the alias becomes a pointer path. + + ``enable`` is ``None`` (all bundles, the default), ``()`` (explicitly + none; the source remains), or a tuple of bundle names. Names match + :data:`~molmcp.components.models.COMPONENT_NAME_PATTERN` and are stored + first-seen unique. Catalog membership is not checked here. + + Attributes: + name: Non-empty, whitespace-free alias chosen by the operator. + locator: Origin as the operator wrote it. + enable: ``None`` means all bundles; ``()`` means none; a non-empty + tuple is bundle names. + + Raises: + ValueError: If ``name`` is empty or carries whitespace, if + ``locator`` is not an accepted locator, or if ``enable`` is not + ``None`` or a sequence of component names. + """ + + name: str + locator: str + enable: tuple[str, ...] | None = None + + def __post_init__(self) -> None: + if not isinstance(self.name, str): + raise ValueError( + f"harness source name must be a string, got {type(self.name).__name__}" + ) + if not self.name: + raise ValueError("a harness source must have a non-empty name") + if any(character.isspace() for character in self.name): + raise ValueError( + f"harness source name must not contain whitespace: {self.name!r}" + ) + if not isinstance(self.locator, str): + raise ValueError( + "harness source locator must be a string, " + f"got {type(self.locator).__name__}" + ) + parse_harness_locator(self.locator) + object.__setattr__(self, "enable", _enable_names(self.enable)) + + @property + def origin_key(self) -> str: + """Canonical origin identity parsed from ``locator``.""" + return parse_harness_locator(self.locator).origin_key + + @property + def ref(self) -> str: + """Git ref from the locator, or ``""``.""" + return parse_harness_locator(self.locator).ref + + @property + def owner(self) -> str: + """Lowercase GitHub owner, or ``""`` for a local locator.""" + return parse_harness_locator(self.locator).owner + + @property + def repo(self) -> str: + """Lowercase GitHub repo without ``.git``, or ``""`` for a local locator.""" + return parse_harness_locator(self.locator).repo + + @property + def path(self) -> str: + """Resolved local path, or ``""`` for a GitHub locator.""" + parsed = parse_harness_locator(self.locator) + return parsed.origin_key if parsed.kind == "local" else "" + + @property + def is_local(self) -> bool: + """Whether ``locator`` named a filesystem path.""" + return parse_harness_locator(self.locator).kind == "local" + + +#: Keys one ``harness`` entry may carry, derived from the dataclass rather than +#: written out: a hand-written literal would silently reject a new field the +#: day someone adds it to :class:`HarnessSource`. +_HARNESS_ENTRY_KEYS: frozenset[str] = frozenset(f.name for f in fields(HarnessSource)) + @dataclass(frozen=True, slots=True) class Settings: @@ -85,6 +219,11 @@ class Settings: python_env: str | None = None discover_include: tuple[str, ...] = () discover_exclude: tuple[str, ...] = () + #: The autonomous harness repositories this install may serve from, in + #: the order the most specific settings file wrote them; the empty tuple + #: is the un-harnessed install. Each entry is a locator plus an alias; + #: identity is the locator's origin key, not the alias. + harness: tuple[HarnessSource, ...] = field(default_factory=tuple) molexp: dict[str, str] = field(default_factory=dict) molq: dict[str, str] = field(default_factory=dict) #: Files that actually contributed, lowest precedence first. @@ -103,6 +242,7 @@ def to_dict(self) -> dict[str, Any]: "pythonEnv": self.python_env, "discoverInclude": list(self.discover_include), "discoverExclude": list(self.discover_exclude), + "harness": [asdict(source) for source in self.harness], "molexp": dict(self.molexp), "molq": dict(self.molq), "layers": [str(path) for path in self.layers], @@ -184,6 +324,7 @@ def load_settings(project_root: str | Path | None = None) -> Settings: ), discover_include=_str_tuple(merged.get("discoverInclude")), discover_exclude=_str_tuple(merged.get("discoverExclude")), + harness=_harness_sources(merged.get("harness") or []), molexp={str(k): str(v) for k, v in (merged.get("molexp") or {}).items()}, molq={str(k): str(v) for k, v in (merged.get("molq") or {}).items()}, layers=tuple(contributing), @@ -194,7 +335,24 @@ def load_settings(project_root: str | Path | None = None) -> Settings: def set_value(path: Path, key: str, value: str) -> dict[str, Any]: - """Set ``key`` (dotted for nested) to a parsed ``value``.""" + """Set ``key`` (dotted for nested) to a parsed ``value``. + + Args: + path: The settings file to edit; created if it does not exist. + key: A top-level key, or ``parent.member`` for a dict-valued setting. + value: The command-line string, coerced to the declared type. + + Returns: + The whole file as written. + + Raises: + SettingsError: If ``key`` names a member of :data:`_OBJECT_LISTS` — a + list of entry objects a string cannot author — or if it is + unknown, unsettable, or ``value`` does not parse. Nothing is + written when it raises: the object-list refusal comes before + :func:`_resolve`, so a refused write leaves no file behind. + """ + _reject_object_list_write(key, leaf="set") root, leaf, container = _resolve(path, key, create=True) container[leaf] = _parse(key, value) write_settings_file(path, root) @@ -202,7 +360,22 @@ def set_value(path: Path, key: str, value: str) -> dict[str, Any]: def add_value(path: Path, key: str, value: str) -> dict[str, Any]: - """Append to a list-valued ``key``, ignoring a duplicate.""" + """Append to a list-valued ``key``, ignoring a duplicate. + + Args: + path: The settings file to edit; created if it does not exist. + key: A list-valued top-level key. + value: The string to append, appended only if not already present. + + Returns: + The whole file as written. + + Raises: + SettingsError: If ``key`` names a member of :data:`_OBJECT_LISTS`, + whose elements are objects rather than strings, or if it is not a + list-valued setting at all. Nothing is written when it raises. + """ + _reject_object_list_write(key, leaf="set") top = key.split(".", 1)[0] if _SCHEMA.get(top) is not list: raise SettingsError(f"{key!r} is not a list-valued setting; use `config set`") @@ -217,7 +390,27 @@ def add_value(path: Path, key: str, value: str) -> dict[str, Any]: def remove_value(path: Path, key: str, value: str | None = None) -> dict[str, Any]: - """Drop ``key`` outright, or one ``value`` from a list-valued key.""" + """Drop ``key`` outright, or one ``value`` from a list-valued key. + + Args: + path: The settings file to edit; it must already carry the key. + key: The key to drop, or the list-valued key to drop ``value`` from. + value: One element to drop, or ``None`` to drop ``key`` itself. + + Returns: + The whole file as written. + + Raises: + SettingsError: If ``key`` is not set in ``path``, if ``value`` is not + in its list, or if ``value`` was given for a member of + :data:`_OBJECT_LISTS`, whose elements are entry objects that a + string cannot address. Only the value arm is guarded — dropping + the whole ``harness`` key is a different operation and keeps + working — and the guard comes before :func:`_resolve`, so a + refused call leaves the file byte for byte as it was. + """ + if value is not None: + _reject_object_list_write(key, leaf="remove") root, leaf, container = _resolve(path, key, create=False) if leaf not in container: raise SettingsError(f"{key!r} is not set in {path}") @@ -232,11 +425,235 @@ def remove_value(path: Path, key: str, value: str | None = None) -> dict[str, An return root +def set_harness_source( + path: Path, + locator: str, + *, + alias: str | None = None, + enable: Sequence[str] = (), + disable: Sequence[str] = (), +) -> dict[str, Any]: + """Upsert one ``harness`` entry, addressed by the locator's origin key. + + Empty ``enable`` / ``disable`` sequences mean "leave the field as it + was"; on insert that is ``None`` (all), which the file records by + omitting the key. ``("all",)`` is a sentinel on either flag, and must + not share the call with named tokens or with the other flag's ``all``. + Named ``enable`` unions an explicit list and replaces the all-sentinel; + named ``disable`` subtracts from an explicit list and is refused while + the field is still all. + + A locator not already configured is appended **last**. ``alias is None`` + names a new source :data:`DEFAULT_HARNESS_ALIAS` (``origin``) and keeps + the stored alias on an update; ``origin`` already taken by another + origin requires ``--alias``. + + Two orderings are the contract. The arguments are validated by + constructing a :class:`HarnessSource` *before* :func:`_resolve`, the way + :func:`set_value` refuses ahead of it, so a refused call leaves no file + behind at all. The merged entry is then constructed a second time, after + the read and still before the write, which is what leaves the dataclass — + never this function — deciding whether the result is legal. + + Args: + path: The settings file to edit; created if it does not exist. + locator: Origin as the operator wrote it; identity is its origin key. + alias: The entry's name, or ``None`` to default on insert and keep + the stored name on update. + enable: Bundle names to turn on, ``("all",)`` for every bundle, or + empty to leave the field as it was. + disable: Bundle names to turn off, ``("all",)`` for none, or empty + to leave the field as it was. + + Returns: + The whole file as written. + + Raises: + SettingsError: If :class:`HarnessSource` refuses the arguments or the + merged entry — carrying the type's own message — if ``all`` is + mixed with named tokens, if named ``disable`` is aimed at the + all-sentinel, if the default alias is taken, or if the file + already on disk fails :func:`read_settings_file`. Nothing is + written when it raises. + """ + enable_tokens = tuple(enable) + disable_tokens = tuple(disable) + enable_all = _is_all_flag(enable_tokens, flag="enable") + disable_all = _is_all_flag(disable_tokens, flag="disable") + if enable_all and disable_all: + raise SettingsError("cannot pass --enable all and --disable all together") + if (enable_all and disable_tokens) or (disable_all and enable_tokens): + raise SettingsError("cannot mix 'all' with named --enable / --disable") + named_enable = () if enable_all else enable_tokens + named_disable = () if disable_all else disable_tokens + if disable_all: + offered_enable: tuple[str, ...] | None = () + elif named_enable or named_disable: + offered_enable = tuple(dict.fromkeys((*named_enable, *named_disable))) + else: + offered_enable = None + offered_name = alias if alias is not None else DEFAULT_HARNESS_ALIAS + _harness_entry({"name": offered_name, "locator": locator, "enable": offered_enable}) + root, leaf, container = _resolve(path, "harness", create=True) + entries: list[dict[str, Any]] = list(container.get(leaf, [])) + sources = _harness_sources(entries) + origin_key = parse_harness_locator(locator).origin_key + at = next( + ( + index + for index, source in enumerate(sources) + if source.origin_key == origin_key + ), + None, + ) + if alias is None: + new_name = DEFAULT_HARNESS_ALIAS if at is None else sources[at].name + else: + new_name = alias + if at is None and alias is None: + if any(source.name == DEFAULT_HARNESS_ALIAS for source in sources): + raise SettingsError( + f"the default harness alias {DEFAULT_HARNESS_ALIAS!r} is " + "already used; pass --alias to name this source" + ) + elif any( + index != at and source.name == new_name for index, source in enumerate(sources) + ): + raise SettingsError(f"harness source name {new_name!r} is already used") + current_enable = None if at is None else sources[at].enable + merged = _harness_entry( + { + "name": new_name, + "locator": locator, + "enable": _merge_enable( + current_enable, + named_enable=named_enable, + named_disable=named_disable, + disable_all=disable_all, + enable_all=enable_all, + ), + } + ) + if at is None: + entries.append(merged) + else: + entries[at] = merged + container[leaf] = entries + write_settings_file(path, root) + return root + + +def match_harness_source( + sources: Sequence[HarnessSource], token: str +) -> HarnessSource | None: + """Return the source whose alias or origin matches ``token``. + + Name is tried first, exact. A token that is also a locator spelling is + still a name if some source uses it as one. Only then is ``token`` + parsed as a locator and compared by ``origin_key``, so a ref is not + identity. + + Args: + sources: Configured harness sources, in file order. + token: An alias, or any accepted locator spelling of an origin. + + Returns: + The first matching source, or ``None`` if none match. + """ + for source in sources: + if source.name == token: + return source + try: + origin_key = parse_harness_locator(token).origin_key + except LocatorError: + return None + for source in sources: + if source.origin_key == origin_key: + return source + return None + + +def remove_harness_source(path: Path, token: str) -> dict[str, Any]: + """Drop the one ``harness`` entry matching ``token``, keeping the rest. + + ``token`` is an alias or a locator spelling; matching is + :func:`match_harness_source`. Removing the last entry leaves + ``"harness": []`` rather than a missing key: dropping the key is + ``remove_value(path, "harness")``, a different operation, and an empty + list is how a file says it named no source. + + Args: + path: The settings file to edit; it must already carry the key. + token: The entry's alias, or any accepted locator spelling of its + origin. + + Returns: + The whole file as written. + + Raises: + SettingsError: If the file has no ``harness`` key, or carries no + entry matching ``token``, or fails :func:`read_settings_file`. + Nothing is written when it raises. + """ + root, leaf, container = _resolve(path, "harness", create=False) + if leaf not in container: + raise SettingsError(f"'harness' is not set in {path}") + entries: list[dict[str, Any]] = container[leaf] + sources = _harness_sources(entries) + matched = match_harness_source(sources, token) + if matched is None: + raise SettingsError(f"{token!r} is not present in 'harness'") + container[leaf] = [ + entry + for entry, source in zip(entries, sources, strict=True) + if source.name != matched.name + ] + write_settings_file(path, root) + return root + + def get_value(data: dict[str, Any], key: str) -> Any: - """Read a dotted ``key`` out of already-parsed settings data.""" + """Read a dotted ``key`` out of already-parsed settings data. + + A key the data does not carry and a path *through* something that is not + an object are different answers, and one condition used to give them the + same one. An undeclared key is ``None``, which reads as "unset"; + ``harness.owner`` is not a path at all now that ``harness`` is a list of + named entries, and answering ``None`` there would say that coordinate is + unset rather than unreachable — the wrong of the two, and the one that + sends an operator looking for a verb to set it with. + + Which case each arm serves is easy to get backwards. + :meth:`Settings.to_dict` carries every key, ``cacheDir`` among them, so a + bare ``cacheDir`` read answers ``None`` because the *value* is ``None`` + and the walk ends — never through the missing-key arm at all. That arm is + reachable only for keys ``to_dict`` does not carry, ``nope`` and + ``sources.nope`` among them. The head is deliberately not checked against + :data:`_SCHEMA` either: ``to_dict`` emits ``layers``, which the schema + does not declare, so validating here would break a read that works. + + Args: + data: One already-merged settings mapping, as + :meth:`Settings.to_dict` renders it. + key: A top-level key, or ``parent.member`` for a nested read. + + Returns: + The value found, or ``None`` if ``key`` names nothing ``data`` carries. + + Raises: + SettingsError: If segments remain but the node they would descend + into is not an object; the message names the whole key and the + segment that is not one. + """ node: Any = data - for part in key.split("."): - if not isinstance(node, dict) or part not in node: + parts = key.split(".") + for index, part in enumerate(parts): + if not isinstance(node, dict): + walked = ".".join(parts[:index]) + raise SettingsError( + f"{key!r} is not a readable path: {walked!r} is not an object" + ) + elif part not in node: return None node = node[part] return node @@ -263,19 +680,159 @@ def _reject_unknown(data: dict[str, Any], path: Path) -> None: f"{', '.join(f'{parent}.{k}' for k in strays)}. " f"Known {parent} keys: {', '.join(sorted(allowed))}" ) + _reject_bad_harness_entries(data, path) + + +def _reject_bad_harness_entries(data: dict[str, Any], path: Path) -> None: + """Check one file's ``harness`` value entry by entry. + + Every rejection is a :class:`SettingsError` naming the file and the + offending entry by position (``harness[1].onwer``), because a list has no + other address to report. The entry rules themselves are not restated here: + each entry is handed to :class:`HarnessSource`, whose ``ValueError`` is + re-raised as a ``SettingsError``, so the type's rules are the only rules. + + Args: + data: One already-parsed settings file. + path: Where it came from, for the message. + + Raises: + SettingsError: If ``harness`` is not a list — a table from the retired + three-key model included — if an element is not an object, carries + a key outside :data:`_HARNESS_ENTRY_KEYS`, carries a retired + coordinate key, omits ``name`` or ``locator``, fails + :class:`HarnessSource` construction, or repeats a ``name`` or + origin key another entry in this same file already used. + """ + if "harness" not in data: + return + entries: list[Any] = data["harness"] + if not isinstance(entries, list): + raise SettingsError( + f"'harness' in {path} must be a list of entry objects " + f"({{{', '.join(sorted(_HARNESS_ENTRY_KEYS))}}}), " + f"not {type(entries).__name__}" + ) + seen_names: set[str] = set() + seen_origins: set[str] = set() + for index, entry in enumerate(entries): + if not isinstance(entry, dict): + raise SettingsError( + f"harness[{index}] in {path} must be an entry object, " + f"not {type(entry).__name__}" + ) + retired = sorted(set(entry) & _RETIRED_HARNESS_KEYS) + if retired: + raise SettingsError( + f"harness[{index}] in {path} uses retired coordinate keys " + f"({', '.join(retired)}); re-run molmcp config harness set " + f"" + ) + strays = sorted(set(entry) - _HARNESS_ENTRY_KEYS) + if strays: + raise SettingsError( + f"unknown setting(s) in {path}: " + f"{', '.join(f'harness[{index}].{k}' for k in strays)}. " + f"Known harness entry keys: {', '.join(sorted(_HARNESS_ENTRY_KEYS))}" + ) + if "name" not in entry: + raise SettingsError(f"harness[{index}] in {path} has no 'name'") + if "locator" not in entry: + raise SettingsError(f"harness[{index}] in {path} has no 'locator'") + try: + source = HarnessSource(**entry) + except ValueError as exc: + raise SettingsError(f"harness[{index}] in {path}: {exc}") from exc + if source.name in seen_names: + raise SettingsError( + f"harness[{index}] in {path} repeats the name {source.name!r}; " + f"harness names are typed by hand and are not renamed for you" + ) + if source.origin_key in seen_origins: + raise SettingsError( + f"harness[{index}] in {path} repeats the origin {source.origin_key!r}" + ) + seen_names.add(source.name) + seen_origins.add(source.origin_key) + + +def _reject_object_list_write(key: str, *, leaf: str) -> None: + """Refuse a string-valued edit verb aimed at a list of entry objects. + + Called first by :func:`set_value`, :func:`add_value` and the value arm of + :func:`remove_value`, before :func:`_resolve` and therefore before + :func:`read_settings_file` and any write. For the two writing verbs, + reaching the write would store ``["x"]`` or append the bare string + ``"x"``, and the per-entry validator then rejects that value on the *next* + read — under ``load_settings``, hence under ``config list``, ``get``, + ``set``, ``remove`` and ``serve`` alike. ``remove_value`` is guarded for + the opposite reason: it corrupts nothing, it compares a string against + entry objects and reports that ``'official'`` is not present while an + entry named ``official`` sits in the file. + + Which keys are refused is read from :data:`_OBJECT_LISTS`, so the next + list-of-objects setting closes this hole by joining that tuple rather than + by someone remembering to add a second branch here. Only the *bare* key is + matched: a dotted ``harness.owner`` cannot equal a top-level table entry + and is already refused by :func:`_resolve`, whose message names the full + key and carries its own pointer at the same verb group. Shadowing that + path here would replace a precise message with a vaguer one. + + The command is **derived** — ``molmcp config {key} {leaf}`` — rather than + written as a ``harness`` literal, so a second member of + :data:`_OBJECT_LISTS` gets a correct message only if it also ships that + verb, which a test asserts rather than a comment promising it. The + ``leaf`` is the *calling* verb's: answering a refused + ``config remove harness official`` with ``... harness set`` would be a + precise misdirection, worse than the vague message it replaces. The shape + sentence enumerates :data:`_HARNESS_ENTRY_KEYS`, the only entry type + declared today; a second member has to generalize that line as it joins. + + Args: + key: The key the caller asked to write, dotted or bare. + leaf: The ``config `` leaf that does the job the caller was + attempting — ``"set"`` for :func:`set_value` and :func:`add_value` + alike, since one entry is authored by name and there is no ``add`` + leaf, and ``"remove"`` for :func:`remove_value`. + + Raises: + SettingsError: If ``key`` is a bare member of :data:`_OBJECT_LISTS`. + """ + if key not in _OBJECT_LISTS: + return + raise SettingsError( + f"{key!r} is a list of entry objects, not of strings, so it cannot be " + f"written one string at a time. Use `molmcp config {key} {leaf}`, " + f"which addresses one entry by name; an entry carries the keys " + f"{{{', '.join(sorted(_HARNESS_ENTRY_KEYS))}}}." + ) def _resolve( path: Path, key: str, *, create: bool ) -> tuple[dict[str, Any], str, dict[str, Any]]: - """Return ``(root, leaf_name, owning_container)`` for a dotted key.""" + """Return ``(root, leaf_name, owning_container)`` for a dotted key. + + A dotted key whose head is a member of :data:`_OBJECT_LISTS` earns one + extra sentence pointing at that key's own verb group. It names no leaf, + because this function cannot see whether :func:`set_value` or + :func:`remove_value` called it; and a head outside the table keeps the + generic message, because pointing ``excludes.foo`` at a harness verb + would be a worse error than the vague one it gets today. + """ parts = key.split(".") if parts[0] not in _SCHEMA: raise SettingsError( f"unknown setting {parts[0]!r}. Known keys: {', '.join(sorted(_SCHEMA))}" ) if len(parts) > 2 or (len(parts) == 2 and _SCHEMA[parts[0]] is not dict): - raise SettingsError(f"{key!r} is not a settable path") + pointer = ( + f"; {parts[0]!r} is a list of named entries, edited one at a " + f"time by `molmcp config {parts[0]}`" + if parts[0] in _OBJECT_LISTS + else "" + ) + raise SettingsError(f"{key!r} is not a settable path{pointer}") allowed = _NESTED_SCHEMA.get(parts[0]) if len(parts) == 2 else None if allowed is not None and parts[1] not in allowed: raise SettingsError( @@ -319,6 +876,126 @@ def _parse(key: str, value: str) -> Any: return value +def _harness_entry(values: dict[str, Any]) -> dict[str, Any]: + """Build one ``harness`` entry, letting the type own every field rule. + + The editing verbs call this both before they read and again on the merged + result, so the rules an operator's message quotes are :class:`HarnessSource`'s + own — restated nowhere. The ``ValueError`` is re-raised carrying its text, + the way :func:`_reject_bad_harness_entries` does for a file on disk; + only the address differs, since a verb knows a name where a file knows a + position. + + Only the operator fields are written. ``enable is None`` (all) omits the + key; ``()`` is written as ``[]``; a named tuple is written as a list. + Derived identity never lands in the file. + + Args: + values: The fields to construct with; an omitted one takes the + dataclass default rather than being invented here. + + Returns: + The entry as a plain dict of operator fields, ``enable`` omitted + when it is ``None``. + + Raises: + SettingsError: If :class:`HarnessSource` refuses ``values``. + """ + operator = {key: values[key] for key in _HARNESS_ENTRY_KEYS if key in values} + try: + entry = asdict(HarnessSource(**operator)) + except ValueError as exc: + raise SettingsError( + f"harness source {operator.get('name', '')!r}: {exc}" + ) from exc + if entry["enable"] is None: + del entry["enable"] + else: + entry["enable"] = list(entry["enable"]) + return entry + + +def _harness_sources(entries: list[Any]) -> tuple[HarnessSource, ...]: + """Build the entry tuple from a ``harness`` value every layer accepted. + + Args: + entries: The merged ``harness`` list. Each layer passed through + :func:`_reject_bad_harness_entries` on the way in, so every + element here is already known to construct. + + Returns: + One :class:`HarnessSource` per element, in file order. + """ + return tuple(HarnessSource(**entry) for entry in entries) + + +def _enable_names(value: Any) -> tuple[str, ...] | None: + """Normalize ``enable`` to ``None`` or a first-seen-unique name tuple.""" + if value is None: + return None + if isinstance(value, str) or not isinstance(value, (list, tuple)): + raise ValueError( + "harness source enable must be a list of names or None, " + f"got {type(value).__name__}" + ) + names: list[str] = [] + seen: set[str] = set() + for token in value: + if not isinstance(token, str): + raise ValueError( + "harness source enable names must be strings, " + f"got {type(token).__name__}" + ) + if COMPONENT_NAME_PATTERN.fullmatch(token) is None: + raise ValueError( + f"harness source enable name {token!r} is not a component name" + ) + if token not in seen: + seen.add(token) + names.append(token) + return tuple(names) + + +def _is_all_flag(tokens: tuple[str, ...], *, flag: str) -> bool: + """Return whether ``tokens`` is the ``all`` sentinel for one flag.""" + if "all" not in tokens: + return False + if tokens != ("all",): + raise SettingsError(f"cannot mix 'all' with named --{flag} tokens: {tokens}") + return True + + +def _merge_enable( + current: tuple[str, ...] | None, + *, + named_enable: tuple[str, ...], + named_disable: tuple[str, ...], + enable_all: bool, + disable_all: bool, +) -> tuple[str, ...] | None: + """Apply one call's enable/disable flags onto the stored field.""" + if enable_all: + result: tuple[str, ...] | None = None + elif named_enable: + if current is None: + result = tuple(dict.fromkeys(named_enable)) + else: + result = tuple(dict.fromkeys((*current, *named_enable))) + else: + result = current + if disable_all: + return () + if named_disable: + if result is None: + raise SettingsError( + "cannot --disable named bundles while enable is all; " + "pass --enable with the names to keep, or --disable all" + ) + drop = set(named_disable) + return tuple(name for name in result if name not in drop) + return result + + def _str_tuple(value: Any) -> tuple[str, ...]: return tuple(dict.fromkeys(str(item) for item in value or ())) @@ -329,16 +1006,21 @@ def _optional_int(value: Any) -> int | None: __all__ = [ "CONFIG_DIR_NAME", + "DEFAULT_HARNESS_ALIAS", "LOCAL_SETTINGS_NAME", "SETTINGS_NAME", + "HarnessSource", "Settings", "SettingsError", "add_value", "get_value", "load_settings", + "match_harness_source", "project_settings_path", "read_settings_file", + "remove_harness_source", "remove_value", + "set_harness_source", "set_value", "settings_layers", "user_settings_path", diff --git a/src/molmcp/skill/SKILL.md b/src/molmcp/skill/SKILL.md new file mode 100644 index 0000000..6c52e24 --- /dev/null +++ b/src/molmcp/skill/SKILL.md @@ -0,0 +1,102 @@ +--- +name: molcrafts +description: > + Computational chemistry with MolCrafts: find the real package API before + writing science code. Covers molecular structure, topology, force fields, + trajectories, RDF/MSD and other analysis, packing/solvation, 3D visualization, + cluster jobs, and experiment workspaces. +when-to-use: > + Load for any computational-chemistry or molecular-simulation task — draw a + molecule, compute an RDF, pack a box, submit a job, scaffold an experiment — + even if the user never says molmcp or MolCrafts. Do not wait for a slash + command. +user-invocable: false +disable-model-invocation: false +metadata: + author: molmcp + short-description: Discover MolCrafts APIs; never hand-roll science kernels. +--- + +# MolCrafts usage + +Managed by `molmcp init`. Do not edit this file. + +The model loads this skill; the user should not have to invoke it. If it +loaded, use the molcrafts MCP connection. + +## If a tool you need is missing + +Stop and tell the user which of the two recoveries applies. Do not hand-roll +science code while a connection is down. + +### 1. The core is down + +No `packages` / `open` / `route` at all: molmcp is not installed or the host +has not started it. + +```bash +pip install molcrafts-molmcp +molmcp init # grok | claude | cursor | codex +``` + +Then enable the `molcrafts` MCP server in the host. + +### 2. The core is up, a namespaced tool is missing + +Core tools answer but `molvis_open` / `molq_list_jobs` / +`molexp_list_projects` is absent. Do **not** install molmcp again. + +First check whether that plane was simply turned off: run `list_planes`. If +the plane is not listed, it may have been omitted with `molmcp init +--disable ` — reopen it by re-running `molmcp init ` without +that flag. + +Otherwise the plane's science package is not installed. Ask the user to +install the one the missing namespace needs: + +| Missing namespace | Package to install | +|-------------------|--------------------| +| `molvis_*` | `pip install molcrafts-molvis` | +| `molq_*` | `pip install molcrafts-molq` | +| `molexp_*` | `pip install molexp` | + +Then restart the host so `molmcp serve` mounts the plane. + +## Find the capability, then call it + +Science methods are not MCP tools. Discover them, then call them in agent +Python (or `molvis_exec` on the composed server). + +1. `packages` — pick sources from summaries. +2. `outline(source=…)` — module tree. +3. `open(ref)` — signature, docstring, examples. Do this before coding. +4. `compose` / `search` / `suggest` — index helpers only. + +`route(task)` says which **provider namespace** a session needs (draw → +molvis, jobs → molq, workspace → molexp). Default `molmcp serve` mounts +those onto this same connection with FastMCP namespaces: `molvis_open`, +`molq_list_jobs`, `molexp_list_projects`. Core tools stay bare (`packages`, +`open`, `route`). + +Use the package that owns the role (structure / IO / analysis → molpy, +packing → molpack, plots → molplot). Do not reimplement RDF, trajectory +IO, packing, or writers. + +## Missing API is a product gap + +`ok=false` / `SYMBOL_NOT_FOUND` / no public API for the step means the +capability is not there. Stop. Report three lines: + +1. Which step is missing. +2. Which package or ref you opened. +3. What came back. + +Then wait. The user may name another package — run `outline` / `open` on +that source; do not skip discovery. Or they may ask to file an issue: use +`gh` against **that science package's** tracker (not molmcp), body = the +three lines. Do not add a workaround unless they explicitly ask. + +## Experiment planning + +Load **molexp-plan** (`/molexp-plan`) when the user wants to plan, design, +or set up an experiment. This skill only covers API discovery. diff --git a/src/molmcp/skill/__init__.py b/src/molmcp/skill/__init__.py new file mode 100644 index 0000000..60d18cb --- /dev/null +++ b/src/molmcp/skill/__init__.py @@ -0,0 +1,6 @@ +"""Shipped host skills; install only via ``molmcp init``. + +Catalog: ``molcrafts`` (API discovery) and ``molexp-plan`` (experiment planner). +""" + +__all__: list[str] = [] diff --git a/src/molmcp/skill/molexp-plan/SKILL.md b/src/molmcp/skill/molexp-plan/SKILL.md new file mode 100644 index 0000000..dea70dd --- /dev/null +++ b/src/molmcp/skill/molexp-plan/SKILL.md @@ -0,0 +1,102 @@ +--- +name: molexp-plan +description: > + Interactive molexp experiment planner for MolCrafts. Decompose a research + intent into a grounded task board, confirm each step with the user, then + scaffold a molexp experiment and write workflow code against opened APIs. + Use when the user wants to plan, design, or set up an experiment, sweep, + screening study, or workflow — or when they run /molexp-plan. +when-to-use: > + Load when the user wants to plan, design, or scaffold a computational + chemistry experiment, sweep, screening study, or workflow. Also when they + run /molexp-plan. +user-invocable: true +disable-model-invocation: false +metadata: + author: molmcp + short-description: Stepwise experiment planner; confirm, then execute that step. +--- + +# molexp-plan + +Managed by `molmcp init`. Do not edit this file. + +Interactive planner. **One step per turn:** propose, wait for confirm, execute +only that step. Discovery rules live in the **molcrafts** skill — open a +symbol before you commit to it. + +## If molcrafts tools are missing + +Stop. Tell the user: + +```bash +pip install molcrafts-molmcp +molmcp init # grok | claude | cursor | codex +``` + +Do not invent science APIs while the connection is down. + +## Hard rules + +- **One step per turn.** Never fill the whole board in one go. +- **No writes before confirm.** Propose in chat; wait for yes / ok / an edit. + An edit revises the proposal; confirm again before executing. +- **Do not close open questions** unless the user closes them. +- **Do not invent APIs.** `ok=false` / `SYMBOL_NOT_FOUND` is a product gap: + report the missing step, the ref you opened, and what came back — then wait. +- **MCP does not run science.** Do not start a molexp run from a tool. Long or + destructive jobs (`molq` submit, deletes) need their own confirm. + +## Turns + +End every planning turn with the proposal and a confirm prompt. Stop there. + +### 1. Frame + +Restate the objective in the user's words. List open questions (keep them +open). Propose. Wait. + +On confirm: write them into the working plan (chat; and `plan.md` once an +experiment folder exists). + +### 2. Focus + +If molexp tools exist: `molexp_list_projects` / `molexp_list_experiments`. +Propose which project and experiment. Wait. + +On confirm: `molexp_add_project` / `molexp_add_experiment` as needed. If +molexp is missing, keep the plan in chat/files and say so. + +### 3. One task + +For the **next** step only: `packages` → `outline` → `open`. Propose **one** +task: id, name, purpose, opened API ref, non-empty acceptance. The user names +the next step — do not force build → simulate → measure. Wait. + +On confirm: append that task to the board in chat and in `plan.md`. Do not +place the rest. + +Repeat until the user says the board is enough. + +### 4. Realize one (only if asked) + +Propose code for **one** confirmed task, using only opened refs. Wait. + +On confirm: write that file. Then `molexp_validate_workflow` on the snippet +when the tool exists. Still no invented symbols. + +### 5. Run one (only if asked) + +Tell the user how (`molexp run`, or the job tool they already confirmed). +Do not submit a job or start a run until they confirm that action. + +## Board + +Each task has: + +- `id`, `name`, `purpose` +- `ref` — the symbol page you opened +- `acceptance` — one or more strings a later check could test + +The board plus objective, open questions, and inferred-vs-stated values **are** +the plan. Write `plan.md` under the experiment when you have a folder. diff --git a/tests/_ast_checks.py b/tests/_ast_checks.py new file mode 100644 index 0000000..53d9f09 --- /dev/null +++ b/tests/_ast_checks.py @@ -0,0 +1,50 @@ +"""Static checks the tests that read production source as data share. + +Some rules are about a habit rather than a result: "no module reads the +environment for configuration" is answered by parsing the module, not by +running it. The walk that answers it was copied into three test modules, +which meant three places to keep in step the day the rule grows a case. +It lives here once instead. + +Not a fixture and not a ``conftest.py`` entry on purpose: these are plain +functions over an :mod:`ast` node, imported by name from any test module +(``tests`` is on pytest's ``pythonpath``). Nothing here imports molmcp, +touches the filesystem, or holds state. +""" + +from __future__ import annotations + +import ast + +#: Attributes of ``os`` that hand a module the process environment. +_ENVIRONMENT_ATTRS: frozenset[str] = frozenset({"environ", "getenv"}) + + +def reads_environment(tree: ast.AST) -> bool: + """Whether *tree* reads ``os.environ`` or ``os.getenv`` anywhere. + + Both spellings count, and so does a bare ``getenv`` that was imported + ``from os``: the import hides the module name, not the read. + + Args: + tree: A parsed module — or any node — to walk. + + Returns: + ``True`` when the walk finds a read of the environment. + + Examples: + >>> reads_environment(ast.parse("import os\\nx = os.environ['A']")) + True + >>> reads_environment(ast.parse("from os import getenv\\nx = getenv('A')")) + True + >>> reads_environment(ast.parse("x = 1")) + False + """ + for node in ast.walk(tree): + if isinstance(node, ast.Attribute) and node.attr in _ENVIRONMENT_ATTRS: + value = node.value + if isinstance(value, ast.Name) and value.id == "os": + return True + if isinstance(node, ast.Name) and node.id == "getenv": + return True + return False diff --git a/tests/conftest.py b/tests/conftest.py index 8e42f45..058cf2a 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -12,6 +12,7 @@ import pytest from molmcp import CollectionIndex, SourceBinding, create_plane +from molmcp import settings as st from molmcp.discovery import DiscoveryConfig from molmcp.discovery.engine import DiscoveryEngine @@ -38,6 +39,15 @@ def server(tmp_path): ) +@pytest.fixture +def home(tmp_path, monkeypatch): + """A ``tmp_path``-rooted ``Path.home``, so no developer's ``~`` is read.""" + fake = tmp_path / "home" + fake.mkdir() + monkeypatch.setattr(st.Path, "home", staticmethod(lambda: fake)) + return fake + + async def call(server, tool: str, args: dict | None = None): """Helper: invoke ``tool`` and return a Python-friendly result.""" result = await server.call_tool(tool, args or {}) diff --git a/tests/discovery/test_github_freshness.py b/tests/discovery/test_github_freshness.py index e552774..0de5264 100644 --- a/tests/discovery/test_github_freshness.py +++ b/tests/discovery/test_github_freshness.py @@ -1,17 +1,19 @@ -"""GitHub ref-freshness tests (network mocked).""" +"""GitHub ref-freshness tests (transport faked).""" from __future__ import annotations import io -import json import tarfile +import pytest + from molmcp.discovery import DiscoveryConfig, DiscoveryEngine from molmcp.discovery.source import github _SHA1 = "a" * 40 _SHA2 = "b" * 40 _FILES = {"calc.py": "def add(a, b):\n return a + b\n"} +_MUL = {"calc.py": "def mul(a, b):\n return a * b\n"} def _make_tarball(top: str, files: dict[str, str]) -> bytes: @@ -25,15 +27,24 @@ def _make_tarball(top: str, files: dict[str, str]) -> bytes: return buf.getvalue() -def fake_http(sha: str, files: dict[str, str]): - def _get(url, token=None, accept="application/vnd.github+json"): - if "codeload" in url: - return _make_tarball(f"repo-{sha}", files) - if "/commits/" in url: - return json.dumps({"sha": sha}).encode("utf-8") - return json.dumps({"default_branch": "main"}).encode("utf-8") +class _FakeTransport: + """GitTransport stand-in: resolve_commit + fetch_archive, no sockets.""" + + def __init__(self, sha: str, files: dict[str, str] | None = None) -> None: + self.sha = sha + self.files = dict(_FILES if files is None else files) + self.archive = _make_tarball(f"repo-{sha}", self.files) + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + return self.sha + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + return self.archive + - return _get +def _install(monkeypatch: pytest.MonkeyPatch, fake: _FakeTransport) -> _FakeTransport: + monkeypatch.setattr(github, "_transport", lambda _config: fake) + return fake def _engine(tmp_path) -> DiscoveryEngine: @@ -45,32 +56,28 @@ def test_freshness_unknown_when_not_indexed(tmp_path): def test_freshness_fresh_after_index(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA1, _FILES)) + _install(monkeypatch, _FakeTransport(_SHA1)) engine = _engine(tmp_path) engine.index("github:owner/repo") assert engine.check_freshness("github:owner/repo") == "fresh" def test_freshness_stale_when_remote_moves(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA1, _FILES)) + _install(monkeypatch, _FakeTransport(_SHA1)) engine = _engine(tmp_path) engine.index("github:owner/repo") - monkeypatch.setattr(github, "_http_get", fake_http(_SHA2, _FILES)) + _install(monkeypatch, _FakeTransport(_SHA2)) assert engine.check_freshness("github:owner/repo") == "stale" def test_refresh_picks_up_new_commit(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA1, _FILES)) + _install(monkeypatch, _FakeTransport(_SHA1)) engine = _engine(tmp_path) first = engine.index("github:owner/repo") assert first.snapshot.commit == _SHA1 - monkeypatch.setattr( - github, - "_http_get", - fake_http(_SHA2, {"calc.py": "def mul(a, b):\n return a * b\n"}), - ) + _install(monkeypatch, _FakeTransport(_SHA2, _MUL)) result = engine.refresh("github:owner/repo") assert result.snapshot.commit == _SHA2 assert result.freshness == "fresh" diff --git a/tests/discovery/test_github_source.py b/tests/discovery/test_github_source.py index 0891c6b..03aa4b5 100644 --- a/tests/discovery/test_github_source.py +++ b/tests/discovery/test_github_source.py @@ -1,18 +1,22 @@ -"""GitHub source resolution tests (network mocked).""" +"""GitHub source resolution tests (transport faked; no DiscoveryEngine).""" from __future__ import annotations import io -import json import tarfile +from pathlib import Path import pytest -from molmcp.discovery import DiscoveryConfig, DiscoveryEngine +from molmcp.components.git import GitError +from molmcp.discovery.cache.snapshotcache import SnapshotCache +from molmcp.discovery.config import DiscoveryConfig from molmcp.discovery.source import SourceError, github +from molmcp.discovery.source.github import latest_commit, resolve_github _SHA = "a" * 40 _FILES = {"calc.py": "def add(a, b):\n return a + b\n"} +_GITHUB_PY = Path(github.__file__).resolve() def _make_tarball(top: str, files: dict[str, str]) -> bytes: @@ -26,61 +30,118 @@ def _make_tarball(top: str, files: dict[str, str]) -> bytes: return buf.getvalue() -def fake_http(sha: str, files: dict[str, str]): - """Build an ``_http_get`` replacement serving a fake repo.""" - - def _get(url, token=None, accept="application/vnd.github+json"): - if "codeload" in url: - return _make_tarball(f"repo-{sha}", files) - if "/commits/" in url: - return json.dumps({"sha": sha}).encode("utf-8") - return json.dumps({"default_branch": "main"}).encode("utf-8") - - return _get - - -def _engine(tmp_path) -> DiscoveryEngine: - return DiscoveryEngine(DiscoveryConfig(cache_dir=tmp_path / "cache")) - - -def test_resolves_ref_to_commit_sha(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA, _FILES)) - result = _engine(tmp_path).index("github:owner/repo") - assert result.snapshot.origin == "github" - assert result.snapshot.commit == _SHA - assert result.snapshot.snapshot_id == f"github:commit:{_SHA}" - - -def test_extracts_graph_from_tarball(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA, _FILES)) - graph = _engine(tmp_path).get_graph("github:owner/repo") - assert "calc.add" in {n.qualname for n in graph.nodes} - - -def test_second_index_is_cache_first(monkeypatch, tmp_path): - calls: list[str] = [] - served = fake_http(_SHA, _FILES) - - def counting(url, token=None, accept="application/vnd.github+json"): - calls.append(url) - return served(url, token, accept) - - monkeypatch.setattr(github, "_http_get", counting) - engine = _engine(tmp_path) - engine.index("github:owner/repo") - after_first = len(calls) - assert after_first > 0 - - engine.index("github:owner/repo") - assert len(calls) == after_first # cache-first: no extra network - - -def test_ref_in_spec_is_recorded(monkeypatch, tmp_path): - monkeypatch.setattr(github, "_http_get", fake_http(_SHA, {"m.py": "x = 1\n"})) - result = _engine(tmp_path).index("github:owner/repo@dev") - assert result.snapshot.ref == "dev" - - -def test_invalid_spec_raises(tmp_path): - with pytest.raises(SourceError): - _engine(tmp_path).index("github:not-a-valid-spec") +class _FakeTransport: + """GitTransport stand-in: resolve_commit + fetch_archive, no sockets.""" + + def __init__( + self, + sha: str = _SHA, + files: dict[str, str] | None = None, + *, + error: BaseException | None = None, + ) -> None: + self.sha = sha + self.files = dict(_FILES if files is None else files) + self.archive = _make_tarball(f"repo-{sha}", self.files) + self.error = error + self.resolve_calls: list[tuple[str, str, str | None]] = [] + self.fetch_calls: list[tuple[str, str, str]] = [] + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + self.resolve_calls.append((owner, repo, ref)) + if self.error is not None: + raise self.error + return self.sha + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + self.fetch_calls.append((owner, repo, sha)) + return self.archive + + +def _config(tmp_path: Path) -> DiscoveryConfig: + return DiscoveryConfig(cache_dir=tmp_path / "cache") + + +def _install(monkeypatch: pytest.MonkeyPatch, fake: _FakeTransport) -> _FakeTransport: + monkeypatch.setattr(github, "_transport", lambda _config: fake) + return fake + + +class TestResolveGithub: + def test_snapshot_identity_inner_tree_and_extracted_marker( + self, monkeypatch, tmp_path + ): + config = _config(tmp_path) + _install(monkeypatch, _FakeTransport()) + snapshot = resolve_github("github:owner/repo", config) + + assert snapshot.snapshot_id == "github:commit:" + _SHA + assert snapshot.commit == _SHA + assert snapshot.origin == "github" + assert snapshot.root_dir.name == f"repo-{_SHA}" + assert snapshot.root_dir.is_dir() + assert any(f.rel_path == "calc.py" for f in snapshot.files) + + marker = SnapshotCache(config).raw_dir(snapshot.snapshot_id) / ".extracted" + assert marker.is_file() + assert marker.read_text(encoding="utf-8").strip() == str(snapshot.root_dir) + + def test_ref_in_spec_is_passed_to_resolve_commit(self, monkeypatch, tmp_path): + fake = _install(monkeypatch, _FakeTransport()) + snapshot = resolve_github("github:owner/repo@dev", _config(tmp_path)) + assert snapshot.ref == "dev" + assert fake.resolve_calls + assert fake.resolve_calls[0] == ("owner", "repo", "dev") + + def test_invalid_spec_does_not_call_transport(self, monkeypatch, tmp_path): + fake = _install(monkeypatch, _FakeTransport()) + with pytest.raises(SourceError): + resolve_github("github:not-a-valid-spec", _config(tmp_path)) + assert fake.resolve_calls == [] + assert fake.fetch_calls == [] + + def test_git_error_is_mapped_to_source_error(self, monkeypatch, tmp_path): + _install( + monkeypatch, + _FakeTransport( + error=GitError("GitHub request failed (404) for https://example") + ), + ) + with pytest.raises(SourceError, match="GitHub request failed") as caught: + resolve_github("github:owner/repo", _config(tmp_path)) + assert isinstance(caught.value, SourceError) + assert not isinstance(caught.value, GitError) + + def test_second_resolve_skips_fetch_archive(self, monkeypatch, tmp_path): + config = _config(tmp_path) + fake = _install(monkeypatch, _FakeTransport()) + resolve_github("github:owner/repo", config) + assert len(fake.fetch_calls) == 1 + resolve_github("github:owner/repo", config) + assert len(fake.fetch_calls) == 1 + + +class TestLatestCommit: + def test_returns_same_sha_as_resolve_github(self, monkeypatch, tmp_path): + config = _config(tmp_path) + _install(monkeypatch, _FakeTransport()) + snapshot = resolve_github("github:owner/repo", config) + assert latest_commit("github:owner/repo", config) == snapshot.commit + assert latest_commit("github:owner/repo", config) == _SHA + + +class TestGithubModuleSource: + def test_does_not_import_urllib(self): + source = _GITHUB_PY.read_text(encoding="utf-8") + assert "import urllib" not in source + assert "urllib." not in source + + def test_drops_legacy_http_and_extract_names(self): + source = _GITHUB_PY.read_text(encoding="utf-8") + for needle in ( + "_http_get", + "resolve_ref", + "_safe_extract", + "tarfile.extractall", + ): + assert needle not in source, needle diff --git a/tests/fixtures/gate/contract-fail/.github/workflows/official-gate.yml b/tests/fixtures/gate/contract-fail/.github/workflows/official-gate.yml new file mode 100644 index 0000000..989178f --- /dev/null +++ b/tests/fixtures/gate/contract-fail/.github/workflows/official-gate.yml @@ -0,0 +1,49 @@ +# Fixture: the workflow half of `contract-fail/` is legal on purpose. +# +# It is character-for-character the `wired/` workflow. The planted breakage +# lives in this tree's `.pre-commit-config.yaml`, whose official-gate +# `entry:` no longer equals the pull-request job's `run:` below — so the +# failure run_gate reports can only be that one disagreement. +name: Official Gate + +on: + pull_request: + branches: [master, dev] + schedule: + - cron: "0 6 * * 1" + workflow_dispatch: + +concurrency: + group: official-gate-${{ github.ref }} + cancel-in-progress: true + +jobs: + official-gate: + name: official/gate + if: github.event_name != 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate + + official-gate-schedule: + name: official/gate (schedule) + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate diff --git a/tests/fixtures/gate/contract-fail/.pre-commit-config.yaml b/tests/fixtures/gate/contract-fail/.pre-commit-config.yaml new file mode 100644 index 0000000..ae73d9f --- /dev/null +++ b/tests/fixtures/gate/contract-fail/.pre-commit-config.yaml @@ -0,0 +1,43 @@ +# Fixture: the planted breakage, and the only difference from `wired/`. +# +# The official-gate hook wraps the call in `bash -c 'uv sync --extra dev && …'`, +# so its `entry:` no longer equals the pull-request job's literal +# `run: uv run molmcp gate`. Everything else in this tree is the legal wiring, +# which is what makes the reported failure attributable to this line. + +default_install_hook_types: [pre-commit, pre-push] + +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v6.0.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + args: ['--unsafe'] + + - repo: local + hooks: + - id: ci-lint + name: "CI lint (same as ci.yml)" + entry: bash -c 'uv sync --extra dev && uv run ruff check src tests && uv run ruff format --check src tests' + language: system + pass_filenames: false + always_run: true + stages: [pre-commit, pre-push] + + - id: ci-test + name: "CI test (same as ci.yml)" + entry: bash -c 'uv sync --extra dev && uv run pytest -v' + language: system + pass_filenames: false + always_run: true + stages: [pre-push] + + - id: official-gate + name: "official/gate (same as official-gate.yml)" + entry: bash -c 'uv sync --extra dev && uv run molmcp gate' + language: system + pass_filenames: false + always_run: true + stages: [pre-push] diff --git a/tests/fixtures/gate/wired/.github/workflows/official-gate.yml b/tests/fixtures/gate/wired/.github/workflows/official-gate.yml new file mode 100644 index 0000000..eaad174 --- /dev/null +++ b/tests/fixtures/gate/wired/.github/workflows/official-gate.yml @@ -0,0 +1,49 @@ +# Fixture: a legal wiring. run_gate(root=) must return ok. +# +# Both jobs run the one literal `uv run molmcp gate`. `uv sync --extra dev` +# is the prior Install step and is not the compared token. The `${{ }}` in +# `concurrency:` is deliberate: expressions are legal everywhere except in a +# `run:`, which is the only place parity reads. +name: Official Gate + +on: + pull_request: + branches: [master, dev] + schedule: + - cron: "0 6 * * 1" + workflow_dispatch: + +concurrency: + group: official-gate-${{ github.ref }} + cancel-in-progress: true + +jobs: + official-gate: + name: official/gate + if: github.event_name != 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate + + official-gate-schedule: + name: official/gate (schedule) + if: github.event_name == 'schedule' + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: astral-sh/setup-uv@v5 + with: + python-version: '3.12' + enable-cache: true + - name: Install + run: uv sync --extra dev + - name: Gate + run: uv run molmcp gate diff --git a/tests/fixtures/gate/wired/.pre-commit-config.yaml b/tests/fixtures/gate/wired/.pre-commit-config.yaml new file mode 100644 index 0000000..b56afc8 --- /dev/null +++ b/tests/fixtures/gate/wired/.pre-commit-config.yaml @@ -0,0 +1,43 @@ +# Fixture: the pre-commit half of a legal wiring. +# +# `official-gate` carries the bare literal as its `entry:` — not `entry: uv` +# plus `args:`, and not a `bash -c 'uv sync && …'` wrapper — so it equals the +# pull-request job's `run:` character for character. It is a pre-push hook; +# the commit stage still holds only the existing ci-lint. + +default_install_hook_types: [pre-commit, pre-push] + +repos: + - repo: https://github.com/pre-commit/pre-commit-hooks + rev: v6.0.0 + hooks: + - id: trailing-whitespace + - id: end-of-file-fixer + - id: check-yaml + args: ['--unsafe'] + + - repo: local + hooks: + - id: ci-lint + name: "CI lint (same as ci.yml)" + entry: bash -c 'uv sync --extra dev && uv run ruff check src tests && uv run ruff format --check src tests' + language: system + pass_filenames: false + always_run: true + stages: [pre-commit, pre-push] + + - id: ci-test + name: "CI test (same as ci.yml)" + entry: bash -c 'uv sync --extra dev && uv run pytest -v' + language: system + pass_filenames: false + always_run: true + stages: [pre-push] + + - id: official-gate + name: "official/gate (same as official-gate.yml)" + entry: uv run molmcp gate + language: system + pass_filenames: false + always_run: true + stages: [pre-push] diff --git a/tests/providers/test_molexp_remote_resolve.py b/tests/providers/test_molexp_remote_resolve.py new file mode 100644 index 0000000..c900cc6 --- /dev/null +++ b/tests/providers/test_molexp_remote_resolve.py @@ -0,0 +1,57 @@ +"""Host-qualified workspace specs (``Arrhenius:/home/…``) for local MCP tools.""" + +from __future__ import annotations + +from pathlib import Path + +import pytest + +pytest.importorskip("molexp") + + +def test_is_host_qualified() -> None: + from molmcp.providers.molexp.resolve import is_host_qualified + + assert is_host_qualified("Arrhenius:/home/jicli594/work/mace-nve") + assert is_host_qualified("user@host:/data/ws") + assert is_host_qualified("login.hpc.example:/scratch/ws") + assert not is_host_qualified("/home/local/ws") + assert not is_host_qualified("https://example.com/ws") + assert not is_host_qualified("C:\\Users\\ws") + assert not is_host_qualified("") + + +def test_validate_workspace_local_still_ok(tmp_path: Path) -> None: + from molexp.workspace import Workspace + + from molmcp.providers.molexp.provider import MolexpProvider + + ws = Workspace(tmp_path / "lab") + ws.materialize() + ws.add_project("p").add_experiment("e").add_run(params={"t": 1}) + + report = MolexpProvider().validate_workspace(str(ws.resolve())) + assert report["ok"] is True + assert report.get("remote") is False + assert report["error_count"] == 0 + + +def test_list_projects_via_live_workspace(tmp_path: Path) -> None: + """Scaffold helpers accept an already-open Workspace (remote-safe).""" + from molexp.workspace import Workspace + + from molmcp.providers.molexp.scaffold import add_project, list_experiments + + ws = Workspace(tmp_path / "lab2") + ws.materialize() + out = add_project(ws, "alpha") + assert out["project_id"] == "alpha" + exp = list_experiments(ws, "alpha") + assert exp == [] + + +def test_provider_init_keeps_host_qualified_string() -> None: + from molmcp.providers.molexp.provider import MolexpProvider + + p = MolexpProvider("Arrhenius:/home/jicli594/work/mace-nve") + assert p._workspace == "Arrhenius:/home/jicli594/work/mace-nve" diff --git a/tests/test_cli_cache.py b/tests/test_cli_cache.py index d956872..f18e698 100644 --- a/tests/test_cli_cache.py +++ b/tests/test_cli_cache.py @@ -11,20 +11,10 @@ import sqlite3 import time -import pytest - from molmcp import cli from molmcp import settings as st -@pytest.fixture -def home(tmp_path, monkeypatch): - fake = tmp_path / "home" - fake.mkdir() - monkeypatch.setattr(st.Path, "home", staticmethod(lambda: fake)) - return fake - - def _config(tmp_path, cache_dir) -> None: """Point this install's cache at a scratch directory.""" st.write_settings_file( diff --git a/tests/test_cli_config.py b/tests/test_cli_config.py index e4e3915..f4ee9ee 100644 --- a/tests/test_cli_config.py +++ b/tests/test_cli_config.py @@ -1,27 +1,28 @@ """`molmcp config` — the CLI half of the settings surface. -Verb shape follows ``claude config``: list / get / set / add / remove. -The scope default is the one deliberate departure — writes land in the -user file unless ``--project`` is passed, because a plane server's working -directory belongs to whichever MCP client launched it. +Verb shape follows ``claude config``: list / get / set / add / remove, +plus ``harness set|remove`` — the one key whose value is a list of objects +gets its own nested pair, because the string verbs take a string and +cannot author an entry. The scope default is the one deliberate departure +— writes land in the user file unless ``--project`` is passed, because a +plane server's working directory belongs to whichever MCP client launched +it. """ from __future__ import annotations +import argparse +import ast +import inspect import json +from pathlib import Path import pytest from molmcp import cli from molmcp import settings as st - - -@pytest.fixture -def home(tmp_path, monkeypatch): - fake = tmp_path / "home" - fake.mkdir() - monkeypatch.setattr(st.Path, "home", staticmethod(lambda: fake)) - return fake +from molmcp.config import AppConfig, ConfigurationError +from molmcp.harness_paths import pointer_path def _user_settings() -> dict: @@ -29,6 +30,77 @@ def _user_settings() -> dict: return json.loads(path.read_text()) if path.is_file() else {} +def _subparser_choices( + parser: argparse.ArgumentParser, +) -> dict[str, argparse.ArgumentParser]: + """The sub-commands ``parser`` registers, by name. + + The one place in this suite that reads argparse internals. Two tests + need the registered ``config`` action names — the dispatch-coverage + test and the ``_OBJECT_LISTS`` obligation — and one private-API + surface is enough for both. A parser that registers no sub-commands + answers ``{}`` rather than raising, so a missing leaf shows up as a + failed assertion instead of a traversal error. + """ + for action in parser._actions: + if isinstance(action, argparse._SubParsersAction): + return dict(action.choices) + return {} + + +def _config_action_parsers() -> dict[str, argparse.ArgumentParser]: + """Every ``molmcp config `` the real parser registers.""" + return _subparser_choices(_subparser_choices(cli._build_parser())["config"]) + + +def _run(argv: list[str]) -> int: + """``cli.main`` for a command that must parse, not argparse-exit.""" + try: + return cli.main(argv) + except SystemExit as exc: + raise AssertionError( + f"cli.main({argv!r}) raised SystemExit({exc.code})" + ) from exc + + +def _option_strings(parser: argparse.ArgumentParser) -> set[str]: + return {flag for action in parser._actions for flag in action.option_strings} + + +def _positional_dests(parser: argparse.ArgumentParser) -> list[str]: + return [ + action.dest + for action in parser._actions + if action.option_strings == [] and action.dest != "help" + ] + + +def _cli_imported_targets() -> tuple[str, ...]: + """Absolute dotted import targets of ``cli.py``, relative imports resolved.""" + path = Path(cli.__file__).resolve() + parts = ["molmcp"] + found: list[str] = [] + for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))): + if isinstance(node, ast.Import): + found.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + base = parts[: len(parts) - (node.level - 1)] + tail = node.module.split(".") if node.module else [] + module = ".".join([*base, *tail]) + else: + module = node.module or "" + found.append(module) + found.extend(f"{module}.{alias.name}" for alias in node.names) + return tuple(found) + + +def _reaches(targets: tuple[str, ...], dotted: str) -> bool: + return any( + target == dotted or target.startswith(f"{dotted}.") for target in targets + ) + + class TestConfigScope: def test_set_writes_the_user_file_by_default(self, home, monkeypatch, tmp_path): monkeypatch.chdir(tmp_path) @@ -71,6 +143,39 @@ def test_list_reports_the_resolved_settings_and_their_layers( assert payload["sources"] == {"molpy": "pkg:molpy"} assert str(st.user_settings_path()) in payload["layers"] + def test_list_prints_harness_as_an_array_of_entry_objects( + self, home, monkeypatch, tmp_path, capsys + ): + """`harness` reaches the terminal as a JSON array, not an object. + + ``Settings.to_dict`` is the second reader of the setting and + ``config list`` prints what it returns, so the list-of-objects + shape is user-visible output rather than an internal detail. + + ``to_dict`` is ``asdict`` over the dataclass, so an entry reports + the three operator fields — including ``enable: None`` when the + file omitted the key. Derived identity (owner, repo, path) is not + a field and does not appear. + """ + monkeypatch.chdir(tmp_path) + assert ( + _run(["config", "harness", "set", "acme/harness", "--alias", "mine"]) == 0 + ) + capsys.readouterr() + + assert cli.main(["config", "list"]) == 0 + + harness = json.loads(capsys.readouterr().out)["harness"] + assert isinstance(harness, list) + assert len(harness) == 1 + assert isinstance(harness[0], dict) + assert set(harness[0]) == {"name", "locator", "enable"} + assert harness[0] == { + "name": "mine", + "locator": "acme/harness", + "enable": None, + } + def test_get_reads_one_key(self, home, monkeypatch, tmp_path, capsys): monkeypatch.chdir(tmp_path) cli.main(["config", "set", "sources.molpy", "pkg:molpy"]) @@ -135,3 +240,436 @@ def test_removing_an_absent_key_is_reported( assert cli.main(["config", "remove", "sources.nope"]) == 2 assert capsys.readouterr().err.startswith("molmcp:") + + +class TestConfigHarness: + """`molmcp config harness set|remove` — the writer for the one object list. + + ``harness`` is a list of named entry objects, so the string verbs + cannot author it: ``set`` refuses the bare key and no dotted path into + an entry exists. These leaves are the CLI's only route to one; the + settings file itself is still the other, and stays the only one for a + file these verbs can no longer read. + + The operator types a locator: ``molmcp config harness set MolCrafts/harness``. + Optional ``--alias``, optional repeatable ``--enable`` / ``--disable``. + Coordinate flags (``--name --owner --repo --ref --path``) are gone. + Renaming an existing origin with ``--alias`` goes through + ``relocate_pointer`` so an activation pointer follows the new name. + """ + + def test_set_writes_the_typed_locator_under_the_default_origin_alias( + self, home, monkeypatch, tmp_path + ): + """The verb drives the real ``settings.set_harness_source``. + + Nothing is monkeypatched, deliberately: a spy standing in for the + writer would keep passing while the file on disk carried a shape + no reader accepts, which is the ``faked-seam-hides-broken-reader`` + failure this exact key has already had once. + + The locator is stored as typed. Identity is derived at load, so + ``owner`` / ``repo`` / ``path`` never become keys in the file. + """ + monkeypatch.chdir(tmp_path) + + assert _run(["config", "harness", "set", "MolCrafts/harness"]) == 0 + + written = _user_settings()["harness"] + assert written == [{"name": "origin", "locator": "MolCrafts/harness"}] + assert "owner" not in written[0] + assert "repo" not in written[0] + assert "path" not in written[0] + assert "ref" not in written[0] + assert "enable" not in written[0] + + def test_alias_names_the_entry(self, home, monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + + assert ( + _run( + [ + "config", + "harness", + "set", + "MolCrafts/harness", + "--alias", + "official", + ] + ) + == 0 + ) + + assert _user_settings()["harness"] == [ + {"name": "official", "locator": "MolCrafts/harness"} + ] + + def test_repeatable_enable_writes_the_named_list(self, home, monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + + assert ( + _run( + [ + "config", + "harness", + "set", + "MolCrafts/harness", + "--enable", + "sci", + "--enable", + "dev", + ] + ) + == 0 + ) + + assert _user_settings()["harness"] == [ + { + "name": "origin", + "locator": "MolCrafts/harness", + "enable": ["sci", "dev"], + } + ] + + def test_repeatable_disable_all_writes_an_empty_enable_list( + self, home, monkeypatch, tmp_path + ): + monkeypatch.chdir(tmp_path) + + assert ( + _run( + [ + "config", + "harness", + "set", + "MolCrafts/harness", + "--disable", + "all", + ] + ) + == 0 + ) + + assert _user_settings()["harness"] == [ + {"name": "origin", "locator": "MolCrafts/harness", "enable": []} + ] + + def test_the_set_parser_takes_a_positional_locator_and_drops_the_coordinates( + self, + ): + """Retired coordinate flags are gone; locator is positional.""" + set_parser = _subparser_choices(_config_action_parsers()["harness"])["set"] + flags = _option_strings(set_parser) + for retired in ("--name", "--owner", "--repo", "--ref", "--path"): + assert retired not in flags + assert "--alias" in flags + assert "--enable" in flags + assert "--disable" in flags + assert "locator" in _positional_dests(set_parser) + + def test_retired_coordinate_flags_are_absent_from_set_help(self, capsys): + """Argparse itself is what refuses the old flags, not the handler.""" + with pytest.raises(SystemExit) as excinfo: + cli.main(["config", "harness", "set", "--help"]) + + assert excinfo.value.code == 0 + help_text = capsys.readouterr().out + for retired in ("--name", "--owner", "--repo", "--ref", "--path"): + assert retired not in help_text + + def test_relocate_pointer_takes_config_and_keyword_only_edits(self): + import molmcp.harness_sync as harness_sync + + assert hasattr(harness_sync, "relocate_pointer") + parameters = inspect.signature(harness_sync.relocate_pointer).parameters + assert list(parameters)[:2] == ["config", "settings_path"] + assert parameters["locator"].kind is inspect.Parameter.KEYWORD_ONLY + assert parameters["name"].kind is inspect.Parameter.KEYWORD_ONLY + + def test_relocate_pointer_renames_the_pointer_file( + self, home, monkeypatch, tmp_path + ): + import molmcp.harness_sync as harness_sync + + assert hasattr(harness_sync, "relocate_pointer") + monkeypatch.chdir(tmp_path) + config = AppConfig.from_dict( + {"schema_version": "2", "cache_dir": str(tmp_path / "cache")}, + workspace_root=tmp_path, + ) + assert config.cache_dir is not None + settings_path = st.user_settings_path() + st.set_harness_source(settings_path, "MolCrafts/harness") + old = pointer_path(config.cache_dir, "origin") + old.parent.mkdir(parents=True, exist_ok=True) + old.write_text("origin-pointer\n", encoding="utf-8") + + harness_sync.relocate_pointer( + config, + settings_path, + locator="MolCrafts/harness", + name="official", + ) + + assert not old.exists() + assert ( + pointer_path(config.cache_dir, "official").read_text(encoding="utf-8") + == "origin-pointer\n" + ) + assert json.loads(settings_path.read_text())["harness"] == [ + {"name": "official", "locator": "MolCrafts/harness"} + ] + + def test_relocate_pointer_refuses_when_the_target_pointer_already_exists( + self, home, monkeypatch, tmp_path + ): + import molmcp.harness_sync as harness_sync + + assert hasattr(harness_sync, "relocate_pointer") + monkeypatch.chdir(tmp_path) + config = AppConfig.from_dict( + {"schema_version": "2", "cache_dir": str(tmp_path / "cache")}, + workspace_root=tmp_path, + ) + assert config.cache_dir is not None + settings_path = st.user_settings_path() + st.set_harness_source(settings_path, "MolCrafts/harness") + old = pointer_path(config.cache_dir, "origin") + new = pointer_path(config.cache_dir, "official") + old.parent.mkdir(parents=True, exist_ok=True) + old.write_text("origin-pointer\n", encoding="utf-8") + new.write_text("already-official\n", encoding="utf-8") + before = settings_path.read_text(encoding="utf-8") + + with pytest.raises((ConfigurationError, st.SettingsError)): + harness_sync.relocate_pointer( + config, + settings_path, + locator="MolCrafts/harness", + name="official", + ) + + assert settings_path.read_text(encoding="utf-8") == before + assert old.read_text(encoding="utf-8") == "origin-pointer\n" + assert new.read_text(encoding="utf-8") == "already-official\n" + + def test_relocate_pointer_renames_the_entry_when_no_pointer_file_exists( + self, home, monkeypatch, tmp_path + ): + import molmcp.harness_sync as harness_sync + + assert hasattr(harness_sync, "relocate_pointer") + monkeypatch.chdir(tmp_path) + config = AppConfig.from_dict( + {"schema_version": "2", "cache_dir": str(tmp_path / "cache")}, + workspace_root=tmp_path, + ) + assert config.cache_dir is not None + settings_path = st.user_settings_path() + st.set_harness_source(settings_path, "MolCrafts/harness") + + harness_sync.relocate_pointer( + config, + settings_path, + locator="MolCrafts/harness", + name="official", + ) + + assert json.loads(settings_path.read_text())["harness"] == [ + {"name": "official", "locator": "MolCrafts/harness"} + ] + assert not pointer_path(config.cache_dir, "origin").exists() + assert not pointer_path(config.cache_dir, "official").exists() + + def test_rename_with_alias_relocates_an_existing_pointer_file( + self, home, monkeypatch, tmp_path + ): + """CLI set with a new ``--alias`` moves ``harness.origin.pointer``.""" + monkeypatch.chdir(tmp_path) + cache = (tmp_path / "cache").resolve() + assert _run(["config", "set", "cacheDir", str(cache)]) == 0 + assert _run(["config", "harness", "set", "MolCrafts/harness"]) == 0 + old = pointer_path(cache, "origin") + old.parent.mkdir(parents=True, exist_ok=True) + old.write_text("origin-pointer\n", encoding="utf-8") + + assert ( + _run( + [ + "config", + "harness", + "set", + "MolCrafts/harness", + "--alias", + "official", + ] + ) + == 0 + ) + + assert not old.exists() + assert pointer_path(cache, "official").read_text(encoding="utf-8") == ( + "origin-pointer\n" + ) + assert _user_settings()["harness"] == [ + {"name": "official", "locator": "MolCrafts/harness"} + ] + + def test_cli_does_not_import_locator_or_harness_paths(self): + """``cli.py`` reaches the namer through ``relocate_pointer``, not itself.""" + imported = _cli_imported_targets() + assert not _reaches(imported, "molmcp.components.locator") + assert not _reaches(imported, "molmcp.harness_paths") + + def test_project_flag_writes_beside_the_project(self, home, monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + + assert _run(["config", "harness", "set", "--project", "acme/harness"]) == 0 + + assert _user_settings() == {} + written = st.project_settings_path(tmp_path) + assert json.loads(written.read_text())["harness"][0]["name"] == "origin" + assert json.loads(written.read_text())["harness"][0]["locator"] == ( + "acme/harness" + ) + + def test_local_flag_writes_the_untracked_override( + self, home, monkeypatch, tmp_path + ): + monkeypatch.chdir(tmp_path) + + assert _run(["config", "harness", "set", "--local", "acme/harness"]) == 0 + + assert _user_settings() == {} + written = st.project_settings_path(tmp_path, local=True) + assert json.loads(written.read_text())["harness"][0]["name"] == "origin" + + def test_the_remove_leaf_takes_the_scope_flags_too( + self, home, monkeypatch, tmp_path + ): + """Both leaves compose with ``_scope_arguments``, not just ``set``.""" + monkeypatch.chdir(tmp_path) + _run(["config", "harness", "set", "--project", "acme/harness"]) + + assert _run(["config", "harness", "remove", "--project", "origin"]) == 0 + + assert _user_settings() == {} + written = st.project_settings_path(tmp_path) + assert json.loads(written.read_text()) == {"harness": []} + + def test_remove_drops_the_entry_by_alias(self, home, monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + _run(["config", "harness", "set", "MolCrafts/harness", "--alias", "official"]) + + assert _run(["config", "harness", "remove", "official"]) == 0 + + assert _user_settings() == {"harness": []} + + def test_remove_drops_the_entry_by_locator(self, home, monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + _run(["config", "harness", "set", "MolCrafts/harness", "--alias", "official"]) + + assert _run(["config", "harness", "remove", "MolCrafts/harness"]) == 0 + + assert _user_settings() == {"harness": []} + + def test_removing_an_unknown_name_is_reported( + self, home, monkeypatch, tmp_path, capsys + ): + monkeypatch.chdir(tmp_path) + _run(["config", "harness", "set", "MolCrafts/harness"]) + capsys.readouterr() + + assert cli.main(["config", "harness", "remove", "nope"]) == 2 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "nope" in err + + def test_get_a_dotted_harness_key_is_an_error_not_null( + self, home, monkeypatch, tmp_path, capsys + ): + """`harness.owner` is a path that cannot exist, so it is not `null`. + + ``harness`` is a list; answering ``null`` for a member read on it + reports "unset" for a coordinate that no spelling of the settings + file could ever set. + """ + monkeypatch.chdir(tmp_path) + + assert cli.main(["config", "get", "harness.owner"]) == 2 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "harness.owner" in err + + def test_an_unhandled_config_action_raises_instead_of_removing( + self, home, monkeypatch, tmp_path + ): + """`_config`'s chain ends in a raise, not in a silent `remove_value`. + + ``config_action`` is ``required=True`` with fixed choices, so an + unknown action cannot reach ``_config`` through ``cli.main`` at + all — argparse exits 2 first. The Namespace is therefore built by + hand and handed straight to the handler, which is the only way to + reach the tail of the chain. The spy is a secondary assertion: the + criterion is that the raise happens. + """ + monkeypatch.chdir(tmp_path) + removed: list[tuple] = [] + monkeypatch.setattr( + st, "remove_value", lambda *call, **kwargs: removed.append(call) + ) + + with pytest.raises(ConfigurationError): + cli._config(argparse.Namespace(config_action="teleport")) + + assert removed == [] + + def test_every_registered_config_action_is_dispatched( + self, home, monkeypatch, tmp_path, capsys + ): + """A subparser landing without a branch is what this catches. + + The action names are derived from the real parser rather than + listed, so a new ``config`` leaf is covered the day it is + registered. Only ``ConfigurationError`` — the terminal raise — + counts as undispatched: a branch that *is* wired fails instead on + ``AttributeError`` for the arguments this bare Namespace does not + carry, and building a full Namespace per action would copy every + subparser's argument shape into this test. + """ + monkeypatch.chdir(tmp_path) + actions = _config_action_parsers() + assert actions, "no `config` sub-commands found; the traversal broke" + + undispatched = [] + for action in actions: + try: + cli._config(argparse.Namespace(config_action=action)) + except ConfigurationError: + undispatched.append(action) + except Exception: + pass + capsys.readouterr() + + assert undispatched == [] + + def test_every_object_list_member_has_a_set_leaf(self): + """`_reject_object_list_write` derives a command; this keeps it real. + + That message names ``molmcp config {key} set`` for every member of + ``_OBJECT_LISTS``, so a second member added without its own verb + would hand the operator a command nothing resolves. Asserting the + member alone is not enough — a member offering only ``remove`` + would satisfy that while the derived sentence stayed false. + """ + actions = _config_action_parsers() + + for member in st._OBJECT_LISTS: + assert member in actions, ( + f"`molmcp config {member} set` is a derived hint with no parser" + ) + assert "set" in _subparser_choices(actions[member]), ( + f"`molmcp config {member}` registers no `set` leaf" + ) diff --git a/tests/test_cli_harness.py b/tests/test_cli_harness.py new file mode 100644 index 0000000..a9e9471 --- /dev/null +++ b/tests/test_cli_harness.py @@ -0,0 +1,896 @@ +"""`molmcp harness sync` and `rollback` — the two verbs that move a pointer. + +``molmcp config harness set`` writes a coordinate and ``molmcp serve`` reads +an activation pointer, and until this verb exists nothing fetches, publishes +or activates in between: ``store.publish``, ``Activation.stage`` and +``Activation.promote`` have no production caller at all, so a configured +source can never become a served one. + +``molmcp harness rollback`` is the other direction along that same pointer. +``sync`` moves it forward and records the SHA it displaced in ``previous``, +which exists for exactly one reason — going back — and until this verb exists +``Activation.rollback`` has no production caller either, so an operator who +synced a harness that turned out worse has no supported way back at all: only +hand-editing a JSON pointer file, which is not a thing a shipped install may +require. + +Its own module rather than more of ``tests/test_cli_config.py``, following the +split already in this suite — ``molmcp cache`` has ``test_cli_cache.py`` and +``molmcp config`` has ``test_cli_config.py``. ``harness`` is a second +top-level verb with its own settings surface, its own on-disk artifacts +(the shared store and one pointer file per source) and its own failure +modes, and folding it into the ``config`` module would put two commands' +fixtures in one file. + +**No network.** Every repository here is built by ``git init`` under +``tmp_path``. The one test that has to prove the *remote* arm picks the +GitHub transport patches that class's two methods and serves the archive out +of a local repository, so even that path opens no socket. +""" + +from __future__ import annotations + +import json +import subprocess +from pathlib import Path + +import pytest + +from molmcp import cli +from molmcp import settings as st +from molmcp.components import ( + Activation, + GitError, + GitHubTransport, + ImmutableGitStore, + LocalGitTransport, +) +from molmcp.harness import SUPPORTED_CAPABILITIES, pointer_path + +#: The smallest ``harness.toml`` ``Activation.stage`` will accept: a catalog +#: is refused outright unless it declares both the ``daily`` and the ``dev`` +#: bundle, so "minimal" is three components, not zero. +_MANIFEST = """\ +requires = ["harness-catalog"] + +[[component]] +kind = "skill" +name = "daily" +path = "skills/daily/SKILL.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.daily"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.daily"] +""" +_SKILL = "# daily\n" +_SCRATCH = "still being edited\n" + +#: The same catalog one commit later: the ``daily`` skill has been rewritten +#: and a second skill declared. Two commits that differ in *both* ways are +#: what makes "the pointer went back" checkable on disk — a rollback that +#: restored only the SHA string would leave the newer text in place, and one +#: that restored a tree but read the newer catalog would still place the row +#: only the newer commit declares. ``review`` sits in no bundle, which the +#: catalog allows: only an unknown *member* is refused. +_REVISED_MANIFEST = ( + _MANIFEST + + """ +[[component]] +kind = "skill" +name = "review" +path = "skills/review/SKILL.md" +""" +) +_REVISED_SKILL = "# daily, rewritten badly\n" +_REVIEW_SKILL = "# review\n" + +#: Identity for the commits made here, passed per invocation so no +#: developer's global git config is read and none is written to ``tmp_path``. +_IDENTITY = ( + "-c", + "user.name=molmcp tests", + "-c", + "user.email=tests@molmcp.invalid", +) + + +# -- a real repository, built here ------------------------------------------ +# +# ``tests/test_components/test_git.py`` builds one the same way. Its helpers +# are private names in a module this change does not touch, so they are +# mirrored rather than imported: a CLI test that breaks when the transport's +# own tests are refactored is coupling this suite does not need, and the +# three subprocess calls are cheaper than the dependency. + + +def _git(root: Path, *args: str) -> str: + """Run one git command inside ``root`` and return its stripped stdout.""" + result = subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + text=True, + ) + return result.stdout.strip() + + +def _git_bytes(root: Path, *args: str) -> bytes: + """Run one git command inside ``root`` and return its raw stdout.""" + return subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + ).stdout + + +def _write(path: Path, text: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def _empty_repo(root: Path) -> Path: + """``git init`` and nothing else: a checkout whose ``HEAD`` resolves to nothing.""" + root.mkdir(parents=True, exist_ok=True) + _git(root, "init", "-q", "--initial-branch=main") + return root + + +def _commit(root: Path, message: str) -> str: + """Commit everything currently in ``root`` and return the new SHA.""" + _git(root, "add", "-A") + _git(root, *_IDENTITY, "commit", "--no-gpg-sign", "-q", "-m", message) + return _git(root, "rev-parse", "HEAD") + + +def _checkout(root: Path) -> tuple[Path, str]: + """A one-commit harness checkout; returns the root and its ``HEAD`` SHA.""" + _empty_repo(root) + _write(root / "harness.toml", _MANIFEST) + _write(root / "skills" / "daily" / "SKILL.md", _SKILL) + return root, _commit(root, "first") + + +def _revise(root: Path) -> str: + """Commit a worse second version of *root*; returns the new SHA. + + The regression an operator would want undone, made real: the file the + catalog already declared is rewritten, the catalog grows a row, and both + land in one commit so a single ``sync`` moves the pointer past them. + """ + _write(root / "harness.toml", _REVISED_MANIFEST) + _write(root / "skills" / "daily" / "SKILL.md", _REVISED_SKILL) + _write(root / "skills" / "review" / "SKILL.md", _REVIEW_SKILL) + return _commit(root, "second") + + +def _archive(root: Path, sha: str) -> bytes: + """The tarball GitHub would serve for ``sha``: one top-level directory.""" + return _git_bytes( + root, "archive", "--format=tar.gz", f"--prefix=harness-{sha}/", sha + ) + + +# -- this install ------------------------------------------------------------ + + +class _Unreachable: + """A ``GitTransport`` for the read-only store the assertions bind. + + ``ImmutableGitStore`` refuses ``None``, and reading a published tree + touches no transport, so anything reached through this one means an + assertion helper started fetching. + """ + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + raise AssertionError("reading the store must not resolve a ref") + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + raise AssertionError("reading the store must not fetch an archive") + + +def _install(cache: Path, *harness: dict[str, str]) -> None: + """Write the user settings file this install syncs from.""" + st.write_settings_file( + st.user_settings_path(), + {"cacheDir": str(cache), "watch": False, "harness": list(harness)}, + ) + + +def _pin_home(home: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Aim the *other* spelling of "the user's home" at the ``home`` fixture. + + That fixture pins :meth:`Path.home`, which is how this package finds home + when it looks it up. It is not how ``~`` is *expanded*: + :meth:`Path.expanduser` delegates to :func:`os.path.expanduser`, which + reads the ``HOME`` / ``USERPROFILE`` environment and never consults + :meth:`Path.home`. A test that pinned only one of the two would leave the + developer's real home reachable through the other. + + Mirrored from ``tests/test_harness.py``'s ``_hermetic_home`` rather than + imported: that is a private name in the module mirroring + ``assert_servable``, and two ``setenv`` calls are cheaper than coupling + this suite to it. + + Args: + home: The ``home`` fixture's tree, already created and already the + answer :meth:`Path.home` gives. + monkeypatch: The test's patcher. + + Returns: + *home*, so a caller can name it in one expression. + """ + monkeypatch.setenv("HOME", str(home)) + monkeypatch.setenv("USERPROFILE", str(home)) + return home + + +def _store(cache: Path) -> ImmutableGitStore: + """The one shared store, at the root ``molmcp.harness`` serves out of. + + ``activated_checkouts`` builds ``ImmutableGitStore(root=/harness)``, + so this is not an arbitrary directory: publishing anywhere else would + leave ``molmcp serve`` unable to find the commit that was just activated. + """ + return ImmutableGitStore(root=cache / "harness", transport=_Unreachable()) + + +def _activation(cache: Path, name: str) -> Activation: + """Bind ``/harness..pointer`` with the real reader.""" + return Activation.bind( + pointer_path(cache, name), + store=_store(cache), + supported_capabilities=SUPPORTED_CAPABILITIES, + ) + + +@pytest.fixture +def cache(home, monkeypatch, tmp_path) -> Path: + """A scratch cache root, with the working directory pointed away from it.""" + work = tmp_path / "work" + work.mkdir() + monkeypatch.chdir(work) + return tmp_path / "cache" + + +@pytest.fixture +def synced_twice(cache, tmp_path) -> tuple[str, str]: + """One local source synced at two commits; returns ``(first, second)``. + + Both syncs are the real verb. A fixture that wrote the pointer JSON + directly would be asserting against its own arithmetic: ``previous`` is + the only thing ``rollback`` has to work with, and it is filled by + ``promote`` during the second sync, so a ``sync`` that stopped recording + the SHA it displaced has to fail here rather than be papered over. + """ + root, first = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + assert cli.main(["harness", "sync", "official"]) == 0 + second = _revise(root) + assert cli.main(["harness", "sync", "official"]) == 0 + return first, second + + +class TestHarnessSync: + """The happy path: fetch, publish, activate — over real artifacts. + + Nothing is faked between the verb and the disk. The store is the real + ``ImmutableGitStore`` at the root ``molmcp serve`` reads, and the pointer + is the real file ``Activation`` binds, because a seam standing in for + either would keep passing while the two commands disagreed about where a + commit lives. + """ + + def test_sync_publishes_head_and_activates_it(self, cache, tmp_path): + """One local source, one command: the commit is served-ready after it. + + A local source is SHA-pinned exactly like a remote one, so the thing + published is a *commit* — ``HEAD`` of the checkout — and the pointer + names that commit rather than the directory. + """ + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) == 0 + + store = _store(cache) + assert store.has(head) + assert (store.tree_path(head) / "harness.toml").read_text() == _MANIFEST + assert ( + store.tree_path(head) / "skills" / "daily" / "SKILL.md" + ).read_text() == (_SKILL) + + def test_sync_leaves_the_named_pointer_file_naming_that_commit( + self, cache, tmp_path + ): + """The pointer is ``/harness.official.pointer`` and it is *promoted*. + + The file name is the contract ``activated_checkouts`` reads by, so it + is asserted literally as well as through the namer. ``staged is None`` + is the other half: a SHA left staged is a SHA nothing serves, which is + indistinguishable from a sync that never ran. + """ + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) == 0 + + pointer = cache / "harness.official.pointer" + assert pointer == pointer_path(cache, "official") + assert pointer.is_file() + activation = _activation(cache, "official") + assert activation.current == head + assert activation.staged is None + assert activation.previous is None + + def test_a_second_sync_with_no_new_commit_republishes_nothing( + self, cache, tmp_path + ): + """Idempotent: same commit, same published directory, same pointer. + + ``previous`` is the sharp assertion. A verb that stages and promotes + unconditionally would leave ``current`` looking right while quietly + overwriting the one SHA ``rollback`` had to return to — the second run + would set ``previous`` to the commit that is already current, and the + install would lose its way back. ``st_ino`` is the other half: the SHA + directory ``publish`` installed is still the one on disk, so nothing + was re-fetched and re-``os.replace``d underneath a running server. + """ + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + assert cli.main(["harness", "sync", "official"]) == 0 + published = (cache / "harness" / "commits" / head).stat().st_ino + pointer = json.loads( + pointer_path(cache, "official").read_text(encoding="utf-8") + ) + + assert cli.main(["harness", "sync", "official"]) == 0 + + activation = _activation(cache, "official") + assert activation.current == head + assert activation.previous is None + assert activation.staged is None + assert (cache / "harness" / "commits" / head).stat().st_ino == published + assert ( + json.loads(pointer_path(cache, "official").read_text(encoding="utf-8")) + == pointer + ) + + def test_a_new_commit_moves_the_pointer_and_keeps_the_previous_sha( + self, cache, tmp_path + ): + """Rollback is why ``previous`` exists; a second sync is what fills it. + + Both trees stay published, and the older one still does *not* carry + the file the newer commit added — so ``rollback`` restores a tree, not + just a name. + """ + root, first = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + assert cli.main(["harness", "sync", "official"]) == 0 + _write(root / "skills" / "spec" / "SKILL.md", "# spec\n") + second = _commit(root, "second") + + assert cli.main(["harness", "sync", "official"]) == 0 + + activation = _activation(cache, "official") + assert activation.current == second + assert activation.previous == first + assert activation.staged is None + store = _store(cache) + assert store.has(first) + assert (store.tree_path(second) / "skills" / "spec" / "SKILL.md").is_file() + assert not (store.tree_path(first) / "skills" / "spec" / "SKILL.md").exists() + + def test_the_working_tree_is_not_what_gets_published(self, cache, tmp_path): + """ "Local" means SHA-pinned, not "whatever is on disk right now". + + This is the property that makes a local source rollbackable at all. If + an uncommitted edit could reach the store, the SHA in the pointer + would name a tree that never existed in the repository, and activating + the same commit twice could serve two different sets of files. + """ + root, head = _checkout(tmp_path / "checkout") + _write(root / "scratch.txt", _SCRATCH) + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) == 0 + + tree = _store(cache).tree_path(head) + assert not (tree / "scratch.txt").exists() + assert (tree / "harness.toml").is_file() + + def test_sync_accepts_a_locator_spelling_of_the_same_origin(self, cache, tmp_path): + """Alias or locator: ``match_harness_source`` is how both verbs address.""" + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", str(root)]) == 0 + + assert _activation(cache, "official").current == head + + +class TestHarnessSyncTransportChoice: + """Origin picks the transport; nothing the operator types does. + + ``HarnessSource`` already refuses an entry that carries both a path and a + coordinate, so the entry's *shape* is a total answer to "where does this + come from". A flag would be a second answer, and two answers to one + question is how an install ends up fetching from a repository nobody + named. + + Both classes are patched on :mod:`molmcp.components.git` where they are + defined, so the assertions hold however the verb imports them. + """ + + @pytest.fixture + def local_transports(self, monkeypatch) -> list[Path]: + """Record every ``LocalGitTransport`` root, leaving behaviour intact. + + ``record`` is annotated exactly as the ``__init__`` it stands in for, + ``root: Path``. Widening it to ``Path | str`` would let this fixture + accept a root the real constructor's signature refuses and hand it + straight on, so the recorded value could be a shape production never + passes and the assertions would be checking a call that cannot happen. + """ + roots: list[Path] = [] + original = LocalGitTransport.__init__ + + def record(self: LocalGitTransport, root: Path) -> None: + roots.append(root) + original(self, root) + + monkeypatch.setattr(LocalGitTransport, "__init__", record) + return roots + + def test_a_local_source_gets_the_local_transport_and_no_other( + self, cache, tmp_path, monkeypatch, local_transports + ): + """Constructed on the source's own ``path``, and GitHub is never spoken to.""" + + def refuse(*args: object, **kwargs: object) -> object: + raise AssertionError("a local source must not reach GitHub") + + monkeypatch.setattr(GitHubTransport, "resolve_commit", refuse) + monkeypatch.setattr(GitHubTransport, "fetch_archive", refuse) + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) == 0 + + assert local_transports == [root] + assert _activation(cache, "official").current == head + + def test_a_home_relative_source_is_rooted_at_the_expanded_checkout( + self, cache, home, monkeypatch, local_transports + ): + """``~/checkout`` is the checkout under home, not a literal ``~`` directory. + + ``assert_servable`` accepts a home-relative ``path`` — home is the + same directory in every session, so the entry names one checkout + rather than a different one per client — which makes this the one + servable spelling that is *not* already the directory to read. + Unexpanded, ``~/checkout`` is an ordinary two-segment relative path + read against whatever working directory the client that launched this + process happened to stand in. + + The published ``HEAD`` is the assertion, not a bare exit code: a + literal ``~`` directory does not exist, so a wrongly-rooted transport + fails at ``git`` and a test asserting only "no traceback" would pass + against the bug. The SHA can only have come from the checkout under + home. + """ + root, head = _checkout(_pin_home(home, monkeypatch) / "checkout") + _install(cache, {"name": "official", "locator": "~/checkout"}) + + assert cli.main(["harness", "sync", "official"]) == 0 + + assert local_transports == [root] + assert _store(cache).has(head) + assert _activation(cache, "official").current == head + + def test_a_remote_source_gets_the_github_transport_and_no_other( + self, cache, tmp_path, monkeypatch, local_transports + ): + """The coordinate arm, with the socket replaced and nothing else. + + The fakes stand exactly where the network would: they are handed the + locator's derived lowercase ``owner``/``repo`` and the ref, and + answer with a commit and an archive built from a repository in + ``tmp_path``. Everything after them — flatten, publish, stage, + promote — is the real code. + """ + root, head = _checkout(tmp_path / "origin") + resolved: list[tuple[str, str, str | None]] = [] + fetched: list[tuple[str, str, str]] = [] + + def resolve( + self: GitHubTransport, owner: str, repo: str, ref: str | None + ) -> str: + resolved.append((owner, repo, ref)) + return head + + def fetch(self: GitHubTransport, owner: str, repo: str, sha: str) -> bytes: + fetched.append((owner, repo, sha)) + return _archive(root, sha) + + monkeypatch.setattr(GitHubTransport, "resolve_commit", resolve) + monkeypatch.setattr(GitHubTransport, "fetch_archive", fetch) + _install( + cache, + { + "name": "official", + "locator": "MolCrafts/harness@main", + }, + ) + + assert cli.main(["harness", "sync", "official"]) == 0 + + assert resolved == [("molcrafts", "harness", "main")] + assert fetched == [("molcrafts", "harness", head)] + assert local_transports == [] + assert _activation(cache, "official").current == head + + def test_a_remote_source_can_be_synced_by_locator( + self, cache, tmp_path, monkeypatch, local_transports + ): + """``MolCrafts/harness`` addresses the same origin as the alias.""" + root, head = _checkout(tmp_path / "origin") + + def resolve( + self: GitHubTransport, owner: str, repo: str, ref: str | None + ) -> str: + return head + + def fetch(self: GitHubTransport, owner: str, repo: str, sha: str) -> bytes: + return _archive(root, sha) + + monkeypatch.setattr(GitHubTransport, "resolve_commit", resolve) + monkeypatch.setattr(GitHubTransport, "fetch_archive", fetch) + _install( + cache, + {"name": "official", "locator": "MolCrafts/harness@main"}, + ) + + assert cli.main(["harness", "sync", "MolCrafts/harness"]) == 0 + + assert local_transports == [] + assert _activation(cache, "official").current == head + + +class TestHarnessSyncErrors: + """Every failure is a sentence on stderr and a non-zero exit. + + A traceback out of the CLI is a bug report about molmcp; what an operator + of a half-configured install needs is the name of the thing that is wrong. + """ + + def test_an_unknown_source_name_lists_the_configured_ones( + self, cache, tmp_path, capsys + ): + """Naming the typo is half the message; naming the alternatives is the rest. + + Sources are addressed by an operator-chosen label, so a + ``ConfigurationError`` that only says "unknown" leaves them to go and + read the settings file to find out what they should have typed. + """ + root, _ = _checkout(tmp_path / "checkout") + _install( + cache, + {"name": "official", "locator": str(root)}, + {"name": "private", "locator": "acme/tooling@trunk"}, + ) + + assert cli.main(["harness", "sync", "ghost"]) != 0 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "ghost" in err + assert "official" in err + assert "private" in err + assert not pointer_path(cache, "official").exists() + + def test_a_ref_that_does_not_resolve_is_reported_not_raised( + self, cache, tmp_path, capsys + ): + """A checkout with no commits: ``HEAD`` names nothing, so git fails. + + The real ``LocalGitTransport`` raises ``GitError`` here, which is a + ``RuntimeError`` and so is *not* in the tuple ``cli.main`` already + catches. This test is red twice over until the verb exists and until + that error is mapped onto the CLI's own register: if it escapes, + ``cli.main`` never returns and this fails as an error rather than an + assertion. + """ + root = _empty_repo(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) != 0 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert not pointer_path(cache, "official").exists() + + def test_a_remote_ref_that_does_not_resolve_surfaces_the_transport_error( + self, cache, capsys, monkeypatch + ): + """The same contract on the coordinate arm, from the transport itself. + + The message the transport wrote is what reaches the operator: a + rewrite here would hide which ref, or which repository, git could not + answer for. + """ + + def refuse( + self: GitHubTransport, owner: str, repo: str, ref: str | None + ) -> str: + raise GitError(f"could not resolve {ref} in {owner}/{repo}") + + monkeypatch.setattr(GitHubTransport, "resolve_commit", refuse) + _install( + cache, + { + "name": "official", + "locator": "molcrafts/harness@nope", + }, + ) + + assert cli.main(["harness", "sync", "official"]) != 0 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "nope" in err + assert not pointer_path(cache, "official").exists() + + def test_a_local_path_that_is_no_checkout_is_reported( + self, cache, tmp_path, capsys + ): + """The verb refuses the same half-authored entry ``molmcp serve`` does. + + A ``path`` naming a directory that is not a repository is the local + analogue of a missing ``ref``. Both commands read the same settings + file, so an entry one of them refuses cannot be one the other syncs. + """ + root = tmp_path / "not-a-repo" + root.mkdir() + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "sync", "official"]) != 0 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "official" in err + assert not pointer_path(cache, "official").exists() + + +class TestHarnessRollback: + """The way back: ``previous`` becomes current, asserted on the real files. + + ``sync`` records the SHA it displaced for one reason, and this is that + reason. Nothing here is faked between the verb and the disk: the pointer + read is the file ``molmcp serve`` binds and the store read is the one it + serves out of, because a seam standing in for either would keep passing + while the operator's install went on serving the commit they asked to + leave. + """ + + def test_rollback_activates_the_commit_the_last_sync_displaced( + self, cache, synced_twice + ): + """A then B then rollback: A is current again, and B is not. + + The raw JSON is asserted beside the bound record because ``active`` + is the field a second process reads; a rollback that moved only an + in-memory record would satisfy the reader that wrote it and nothing + else. + + ``previous is None`` afterwards is not incidental — it is the record + transition ``Activation.rollback`` performs (previous → current, + previous cleared, staged untouched) and therefore the answer to what + a *second* rollback can do. It is pinned here so the CLI cannot + quietly acquire a different one, and exercised in + :class:`TestHarnessRollbackErrors`. + """ + first, second = synced_twice + assert _activation(cache, "official").current == second + + assert cli.main(["harness", "rollback", "official"]) == 0 + + pointer = json.loads( + pointer_path(cache, "official").read_text(encoding="utf-8") + ) + assert pointer["active"] == first + activation = _activation(cache, "official") + assert activation.current == first + assert activation.previous is None + assert activation.staged is None + + def test_rollback_accepts_a_locator_spelling_of_the_same_origin( + self, cache, tmp_path + ): + root, first = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + assert cli.main(["harness", "sync", "official"]) == 0 + _revise(root) + assert cli.main(["harness", "sync", "official"]) == 0 + + assert cli.main(["harness", "rollback", str(root)]) == 0 + + assert _activation(cache, "official").current == first + + def test_the_restored_commit_is_still_a_readable_tree_in_the_store( + self, cache, synced_twice + ): + """Why ``previous`` is worth keeping: the tree it names never left. + + A pointer holds names, not trees, so "rolled back" is only true if + the older commit is still published and still holds the older files. + Both are checked, and so is the newer commit — ``rollback`` moves a + pointer and prunes nothing, which is what makes a re-``sync`` forward + a no-op rather than a fetch. + """ + first, second = synced_twice + + assert cli.main(["harness", "rollback", "official"]) == 0 + + store = _store(cache) + assert store.has(first) + assert (store.tree_path(first) / "harness.toml").read_text() == _MANIFEST + assert ( + store.tree_path(first) / "skills" / "daily" / "SKILL.md" + ).read_text() == (_SKILL) + assert not (store.tree_path(first) / "skills" / "review" / "SKILL.md").exists() + assert store.has(second) + assert ( + store.tree_path(second) / "skills" / "daily" / "SKILL.md" + ).read_text() == (_REVISED_SKILL) + + def test_init_places_the_components_of_the_rolled_back_commit( + self, cache, home, monkeypatch, synced_twice + ): + """The point of the verb: the host follows the pointer. + + ``molmcp init`` is run once, *after* the rollback, so every file + under the host directory was placed by a run that read the restored + pointer. Both halves of the claim are checkable that way: the + ``daily`` skill carries the older commit's bytes rather than the ones + that prompted the rollback, and the row only the newer catalog + declares was never even resolved. Running init before the rollback as + well would prove neither — placement replaces destinations and + removes none, so the newer file would still be sitting there. + + The plane list is pinned because ``init`` renders its MCP JSON from + whatever providers this machine can import, which is a fact about the + developer's environment rather than about the pointer under test. + """ + monkeypatch.setattr( + "molmcp.client_config.default_plane_ids", + lambda: ("molcrafts", "molvis"), + ) + + assert cli.main(["harness", "rollback", "official"]) == 0 + assert cli.main(["init", "claude"]) == 0 + + skills = home / ".claude" / "skills" + assert (skills / "daily" / "SKILL.md").read_text() == _SKILL + assert not (skills / "review" / "SKILL.md").exists() + + +class TestHarnessRollbackErrors: + """Nothing to go back to is an ordinary install state, not a crash. + + ``NothingToRollbackError`` is an ``ActivationError``, which is a plain + ``Exception``: it is in none of the types ``cli.main`` funnels + (``ConfigurationError``, ``FileNotFoundError``, ``ValueError``, + ``SettingsError``, ``sqlite3.Error``, ``OSError``, ``GitError``), so + left alone it reaches the operator as a traceback — the same gap + ``GitError`` had to be registered to close on the sync side. Whether the + verb converts it or the funnel registers it is the implementation's + choice; that the funnel is reached, and answers with its exit code, is + not, so ``2`` is pinned rather than "non-zero". + """ + + def test_a_source_synced_exactly_once_has_no_commit_to_return_to( + self, cache, tmp_path, capsys + ): + """One sync fills ``current`` and leaves ``previous`` empty. + + The pointer is asserted afterwards as well: a verb that reported the + refusal but had already written a record would leave the install + activating nothing at all, which is worse than the state it refused. + """ + root, head = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + assert cli.main(["harness", "sync", "official"]) == 0 + + assert cli.main(["harness", "rollback", "official"]) == 2 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "official" in err + # One concern spelled the two ways it can be spelled: the English and + # the verb's own name. Not a disjunction of behaviours. + assert "roll back" in err.lower() or "rollback" in err.lower() + activation = _activation(cache, "official") + assert activation.current == head + assert activation.previous is None + + def test_a_source_that_was_never_synced_writes_no_pointer_file( + self, cache, tmp_path, capsys + ): + """A configured source is not a synced one, and refusing must not create one. + + The missing pointer *is* the empty record, so this reaches the same + refusal by a different road — and the file must still be missing + afterwards, because a pointer written here would be one + ``molmcp init`` and ``molmcp serve`` then have to read. + """ + root, _ = _checkout(tmp_path / "checkout") + _install(cache, {"name": "official", "locator": str(root)}) + + assert cli.main(["harness", "rollback", "official"]) == 2 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "official" in err + assert not pointer_path(cache, "official").exists() + + def test_a_second_rollback_is_refused_rather_than_returning_to_the_newer_commit( + self, cache, synced_twice + ): + """``rollback`` is one level deep, not a toggle between two commits. + + ``Activation.rollback`` clears ``previous`` as it restores it, so + after A→B→rollback there is no recorded way *forward*: the second + call finds ``previous is None`` and refuses exactly as a + never-synced source does. That is the behaviour an operator hits when + they type the command twice, so it is pinned rather than assumed — + the alternative reading, that a second rollback returns to B, would + make the verb a switch and would need a record transition this + install does not have. + + The way back to B is a fresh ``sync``, and the assertion that this is + possible is the last one: B's tree is still published, so that sync + re-fetches nothing. + """ + first, second = synced_twice + assert cli.main(["harness", "rollback", "official"]) == 0 + + assert cli.main(["harness", "rollback", "official"]) == 2 + + activation = _activation(cache, "official") + assert activation.current == first + assert activation.previous is None + assert activation.staged is None + assert _store(cache).has(second) + + def test_an_unknown_source_name_lists_the_configured_ones( + self, cache, tmp_path, capsys + ): + """The same refusal ``sync`` gives, because it is the same question. + + Both verbs address a source by an operator-chosen label out of the + same settings list, so "unknown source" on one and a helpful sentence + on the other would make which command was typed decide whether the + operator learns what they should have typed. + """ + root, _ = _checkout(tmp_path / "checkout") + _install( + cache, + {"name": "official", "locator": str(root)}, + {"name": "private", "locator": "acme/tooling@trunk"}, + ) + + assert cli.main(["harness", "rollback", "ghost"]) != 0 + + err = capsys.readouterr().err + assert err.startswith("molmcp:") + assert "ghost" in err + assert "official" in err + assert "private" in err + assert not pointer_path(cache, "official").exists() diff --git a/tests/test_cli_vnext.py b/tests/test_cli_vnext.py index 27b8a48..d9199cd 100644 --- a/tests/test_cli_vnext.py +++ b/tests/test_cli_vnext.py @@ -1,8 +1,12 @@ from __future__ import annotations +import ast import json +from pathlib import Path -from molmcp import cli +import pytest + +from molmcp import __version__, cli, gate from molmcp.environment import EnvironmentReport @@ -37,14 +41,47 @@ def _config(tmp_path): return path +def test_version_flag(capsys): + with pytest.raises(SystemExit) as exited: + cli.main(["--version"]) + assert exited.value.code == 0 + assert __version__ in capsys.readouterr().out + + +def test_version_short_flag(capsys): + with pytest.raises(SystemExit) as exited: + cli.main(["-V"]) + assert exited.value.code == 0 + assert capsys.readouterr().out.strip() == f"molmcp {__version__}" + + def test_no_arguments_defaults_to_planes(monkeypatch, tmp_path, capsys): monkeypatch.chdir(tmp_path) assert cli.main([]) == 0 out = capsys.readouterr().out - assert "multi-link" in out.lower() or "planes" in out.lower() or "catalog" in out + assert "molcrafts" in out.lower() + assert "catalog is not a plane" not in out.lower() + + +def test_serve_no_plane_uses_stack(monkeypatch, tmp_path): + captured = {} + + class FakeServer: + def run(self, **kwargs): + captured.update(kwargs) + + def fake_stack(**kwargs): + captured["disable"] = list(kwargs.get("disable") or []) + return FakeServer() + + monkeypatch.chdir(tmp_path) + monkeypatch.setattr(cli, "create_stack", fake_stack) + assert cli.main(["serve", "--disable", "molq"]) == 0 + assert captured["disable"] == ["molq"] + assert captured["transport"] == "stdio" -def test_serve_requires_plane(monkeypatch, tmp_path, capsys): +def test_serve_core(monkeypatch, tmp_path, capsys): captured = {} class FakeServer: @@ -53,7 +90,8 @@ def run(self, **kwargs): monkeypatch.chdir(tmp_path) monkeypatch.setattr(cli, "create_plane", lambda *a, **kwargs: FakeServer()) - assert cli.main(["serve", "catalog"]) == 0 + monkeypatch.setattr(cli, "create_stack", lambda **kwargs: FakeServer()) + assert cli.main(["serve", "molcrafts"]) == 0 assert captured == { "transport": "stdio", "show_banner": False, @@ -61,6 +99,18 @@ def run(self, **kwargs): } +def test_serve_catalog_is_user_error(monkeypatch, tmp_path, capsys): + class FakeServer: + def run(self, **kwargs): + raise AssertionError("must fail before run") + + monkeypatch.chdir(tmp_path) + monkeypatch.setattr(cli, "create_plane", lambda *a, **kwargs: FakeServer()) + code = cli.main(["serve", "catalog"]) + assert code == 2 + assert "catalog is not a plane" in capsys.readouterr().err + + def test_search_emits_json(monkeypatch, tmp_path, capsys): class Hit: def to_dict(self): @@ -92,10 +142,11 @@ def run(self, **kwargs): raise AssertionError("must fail before run") monkeypatch.setattr(cli, "create_plane", lambda *a, **kwargs: FakeServer()) + monkeypatch.setattr(cli, "create_stack", lambda **kwargs: FakeServer()) code = cli.main( [ "serve", - "catalog", + "molcrafts", "--config", str(_config(tmp_path)), "--transport", @@ -116,3 +167,204 @@ def test_route_cli(capsys): assert cli.main(["route", "draw a molecule"]) == 0 payload = json.loads(capsys.readouterr().out) assert any(m["plane"] == "molvis" for m in payload["planes"]) + + +# -- `molmcp gate`: dispatch, and nothing else -------------------------- +# +# The verdict has one owner, `molmcp.gate.run_gate`. What is tested here is +# the seam between the two: which root the CLI hands over, which exit code it +# turns the report into, and that it decides nothing on its own. `run_gate` +# is monkeypatched by the name `cli` resolves, the same handling +# `create_stack` gets above — a CLI test that read the real repository would +# be testing gate.py a second time, from further away. + +_CLI_SOURCE = Path(cli.__file__) + +#: Strings belonging to the verdict. Any of them spelled inside `_gate` means +#: the CLI has started re-deriving what gate.py already decided, and the two +#: copies can then disagree about the one required check. +_VERDICT_TOKENS = ( + "official/gate", + "official-gate", + "uv run molmcp gate", + ".pre-commit-config.yaml", + ".github/workflows", + "${{", + "stages", +) + +#: Names that would hand the CLI a second copy of a pinned literal. +_VERDICT_IMPORTS = ("CHECK_NAME", "GATE_RUN", "PR_JOB_ID", "SCHEDULE_JOB_ID") + +#: Flags the gate subparser must not grow. There is one profile: an +#: evaluation needs two subagents and a GitHub runner has none, so a `--full` +#: could never run where it would be wired, and `--skip` is a required check +#: with an off switch. +_REJECTED_FLAGS = ("--full", "--skip") + + +def _patch_gate(monkeypatch, *, ok, failed=()): + """Replace the `run_gate` the CLI resolves; return what it was called with.""" + recorded: dict[str, object] = {} + report = gate.GateReport(ok=ok, failed=failed) + + def run_gate(**kwargs): + recorded.update(kwargs) + return report + + monkeypatch.setattr(cli, "run_gate", run_gate) + return recorded + + +def _gate_handler(): + """The `_gate` handler read as source, or a readable failure.""" + tree = ast.parse(_CLI_SOURCE.read_text(encoding="utf-8")) + handlers = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.FunctionDef) and node.name == "_gate" + ] + assert handlers, ( + "src/molmcp/cli.py defines no `_gate` handler. `molmcp gate` is a " + "dispatch: the handler calls run_gate and prints what it returns; the " + "verdict stays in molmcp.gate." + ) + return handlers[0] + + +def _gate_strings(): + """Every string literal in `_gate`, its own docstring excepted.""" + body = _gate_handler().body + first = body[0] if body else None + if ( + isinstance(first, ast.Expr) + and isinstance(first.value, ast.Constant) + and isinstance(first.value.value, str) + ): + body = body[1:] + return [ + node.value + for statement in body + for node in ast.walk(statement) + if isinstance(node, ast.Constant) and isinstance(node.value, str) + ] + + +def _gate_imports(): + """Every name `cli.py` imports from the gate module.""" + tree = ast.parse(_CLI_SOURCE.read_text(encoding="utf-8")) + return { + alias.name + for node in ast.walk(tree) + if isinstance(node, ast.ImportFrom) and (node.module or "").endswith("gate") + for alias in node.names + } + + +def test_gate_calls_run_gate_with_the_working_directory(monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + recorded = _patch_gate(monkeypatch, ok=True) + + cli.main(["gate"]) + + assert recorded == {"root": Path.cwd()} + + +def test_gate_returns_zero_when_the_report_is_ok(monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + _patch_gate(monkeypatch, ok=True) + + assert cli.main(["gate"]) == 0 + + +def test_gate_returns_one_when_the_report_is_not_ok(monkeypatch, tmp_path): + monkeypatch.chdir(tmp_path) + _patch_gate(monkeypatch, ok=False, failed=("a job runs something else",)) + + assert cli.main(["gate"]) == 1 + + +def test_gate_prints_every_reported_failure(monkeypatch, tmp_path, capsys): + message = "a hook entry: was wrapped and no longer equals the job's run:" + monkeypatch.chdir(tmp_path) + _patch_gate(monkeypatch, ok=False, failed=(message,)) + + cli.main(["gate"]) + + captured = capsys.readouterr() + assert message in captured.out + captured.err + + +@pytest.mark.parametrize("flag", _REJECTED_FLAGS) +def test_gate_subparser_takes_no_flags(monkeypatch, tmp_path, capsys, flag): + monkeypatch.chdir(tmp_path) + _patch_gate(monkeypatch, ok=True) + + with pytest.raises(SystemExit) as exited: + cli.main(["gate", flag]) + + assert exited.value.code != 0 + assert "unrecognized arguments" in capsys.readouterr().err + + +def test_cli_resolves_run_gate_as_its_own_attribute(): + assert cli.run_gate is gate.run_gate, ( + "cli.py must import run_gate into its own namespace " + "(`from .gate import run_gate`), the way it imports create_stack: that " + "is the name the dispatch resolves and the name a test replaces." + ) + + +def test_cli_imports_no_pinned_literal_from_the_gate_module(): + held = sorted(_gate_imports().intersection(_VERDICT_IMPORTS)) + + assert held == [], ( + f"cli.py imports {held} from the gate module. gate.py is the authority " + f"for those literals and the YAML files are its copies; a third copy " + f"in the CLI is one more thing to keep in step." + ) + + +def test_gate_handler_calls_run_gate(): + called = { + node.func.id + for node in ast.walk(_gate_handler()) + if isinstance(node, ast.Call) and isinstance(node.func, ast.Name) + } + + assert "run_gate" in called, ( + f"`_gate` calls {sorted(called)} and never run_gate. The subcommand " + f"exists to ask gate.py for a verdict." + ) + + +def test_gate_handler_derives_no_verdict_of_its_own(): + derived = [ + node + for node in ast.walk(_gate_handler()) + if isinstance(node, (ast.Compare, ast.BoolOp)) + ] + + assert derived == [], ( + f"`_gate` holds {len(derived)} comparison(s) of its own, first on line " + f"{derived[0].lineno if derived else 0}. `ok` is decided once, by " + f"run_gate; a CLI that re-derives it from `failed` is a second verdict " + f"that can disagree with the first. Read `report.ok`." + ) + + +def test_gate_handler_spells_no_verdict_string(): + spelled = sorted( + { + token + for value in _gate_strings() + for token in _VERDICT_TOKENS + if token in value + } + ) + + assert spelled == [], ( + f"`_gate` spells {spelled}. Those are the tokens the report is written " + f"in, and gate.py already names the offending file and token in every " + f"message; the CLI prints what it is handed." + ) diff --git a/tests/test_client_config.py b/tests/test_client_config.py index 95c76b1..3030b83 100644 --- a/tests/test_client_config.py +++ b/tests/test_client_config.py @@ -1,93 +1,119 @@ -"""Client config: default all planes; --enable / --disable.""" +"""Init: one composed serve entry; skill only via init; --disable providers.""" from __future__ import annotations +import ast import json +import re import sys +from pathlib import Path import pytest from molmcp import client_config +from molmcp import host as host_package +from molmcp import skill as skill_package from molmcp.client_config import ( - render_client, + render_init, render_mcp_json, resolve_plane_toggles, ) +from molmcp.planes import CORE_PLANE_ID as CORE -def test_default_all_enabled(): - t = resolve_plane_toggles(available=("catalog", "molcrafts", "molvis", "molq")) - assert t.enabled == ("catalog", "molcrafts", "molvis", "molq") +def test_default_core_plus_providers(): + t = resolve_plane_toggles(available=("molcrafts", "molvis", "molq")) + assert t.enabled == ("molcrafts", "molvis", "molq") assert t.disabled == () def test_disable_then_enable(): t = resolve_plane_toggles( - available=("catalog", "molvis", "molq"), + available=("molcrafts", "molvis", "molq"), disable=["molq", "molvis"], enable=["molvis"], ) - assert t.enabled == ("catalog", "molvis") + assert t.enabled == ("molcrafts", "molvis") assert t.disabled == ("molq",) -def test_disable_unknown_raises(): - with pytest.raises(ValueError, match="unknown plane"): - resolve_plane_toggles(available=("catalog",), disable=["nope"]) +def test_disable_core_raises(): + with pytest.raises(ValueError, match="cannot be disabled"): + resolve_plane_toggles(available=("molcrafts", "molvis"), disable=["molcrafts"]) -def test_disable_all_raises(): - with pytest.raises(ValueError, match="at least one"): - resolve_plane_toggles(available=("a", "b"), disable=["a", "b"]) +def test_disable_catalog_raises(): + with pytest.raises(ValueError, match="catalog is not a plane"): + resolve_plane_toggles(available=("molcrafts", "molvis"), disable=["catalog"]) -def test_a_disabled_plane_is_omitted_from_the_server_map(): - t = resolve_plane_toggles( - available=("catalog", "molvis"), - disable=["molvis"], - ) +def test_composed_server_map_is_a_single_serve(): + t = resolve_plane_toggles(available=("molcrafts", "molvis", "molq")) servers = render_mcp_json(t)["mcpServers"] - assert set(servers) == {"catalog"} - assert servers["catalog"]["args"][-2:] == ["serve", "catalog"] + assert set(servers) == {CORE} + args = servers[CORE]["args"] + assert args[-1] == "serve" or "serve" in args + assert "--disable" not in args -def test_render_client_claude_only_enabled(): - _toggle, text = render_client( - "claude", - disable=["molq"] if False else [], +def test_disabled_provider_becomes_a_serve_flag(): + t = resolve_plane_toggles( + available=("molcrafts", "molvis", "molq"), + disable=["molq"], ) - # smoke: valid JSON with mcpServers - import json + args = render_mcp_json(t)["mcpServers"][CORE]["args"] + assert args[args.index("--disable") + 1] == "molq" + assert "molq" not in t.enabled + +def test_render_init_includes_core(): + _toggle, text = render_init("grok", available=("molcrafts", "molvis")) payload = json.loads(text) - assert "mcpServers" in payload - assert payload["mcpServers"] + assert set(payload["mcpServers"]) == {CORE} -def test_cli_client_disable(capsys, monkeypatch): +def test_cli_init_writes_json_and_skill(tmp_path, monkeypatch, capsys): from molmcp import cli + monkeypatch.setattr(client_config.Path, "home", classmethod(lambda cls: tmp_path)) monkeypatch.setattr( "molmcp.client_config.default_plane_ids", - lambda: ("catalog", "molvis", "molq"), + lambda: ("molcrafts", "molvis", "molq"), ) - # re-import resolve path uses default_plane_ids via resolve_plane_toggles - code = cli.main(["client", "grok", "--disable", "molq"]) + code = cli.main(["init", "grok", "--disable", "molq"]) assert code == 0 - servers = json.loads(capsys.readouterr().out)["mcpServers"] - assert "molq" not in servers - assert set(servers) == {"catalog", "molvis"} - + err = capsys.readouterr().err + assert "wrote" in err + servers = json.loads((tmp_path / ".mcp.json").read_text(encoding="utf-8"))[ + "mcpServers" + ] + assert set(servers) == {CORE} + skill = tmp_path / ".grok" / "skills" / "molcrafts" / "SKILL.md" + assert skill.is_file() + assert "SYMBOL_NOT_FOUND" in skill.read_text(encoding="utf-8") + plan = tmp_path / ".grok" / "skills" / "molexp-plan" / "SKILL.md" + assert plan.is_file() + plan_text = plan.read_text(encoding="utf-8") + assert "One step per turn" in plan_text + assert "No writes before confirm" in plan_text + assert "SYMBOL_NOT_FOUND" in plan_text + assert "molexp-plan" in err + + +def test_cli_init_cannot_disable_core(capsys, monkeypatch, tmp_path): + from molmcp import cli -class TestLaunchableFromAGuiClient: - """A generated config has to start when the client is not a shell. + monkeypatch.setattr(client_config.Path, "home", classmethod(lambda cls: tmp_path)) + monkeypatch.setattr( + "molmcp.client_config.default_plane_ids", + lambda: ("molcrafts", "molvis"), + ) + code = cli.main(["init", "grok", "--disable", "molcrafts"]) + assert code == 2 + assert "cannot be disabled" in capsys.readouterr().err - Claude Desktop and friends are launched by the desktop session, whose - PATH is the system default — a virtualenv's bin directory is not on it. - Emitting the bare name `molmcp` produced a config that worked when - tested in a terminal and failed for the user it was generated for. - """ +class TestLaunchableFromAGuiClient: def test_command_is_the_resolved_absolute_path(self, monkeypatch, tmp_path): installed = tmp_path / "venv" / "bin" / "molmcp" installed.parent.mkdir(parents=True) @@ -95,13 +121,12 @@ def test_command_is_the_resolved_absolute_path(self, monkeypatch, tmp_path): monkeypatch.setattr(client_config.shutil, "which", lambda name: str(installed)) config = client_config.render_mcp_json( - client_config.PlaneToggle(("catalog",), (), ("catalog",)) + client_config.PlaneToggle(("molcrafts",), (), ("molcrafts",)) ) - assert config["mcpServers"]["catalog"]["command"] == str(installed) + assert config["mcpServers"]["molcrafts"]["command"] == str(installed) def test_fallback_uses_this_interpreter_not_a_bare_python(self, monkeypatch): - """`python` is frequently absent on macOS; sys.executable never is.""" monkeypatch.setattr(client_config.shutil, "which", lambda name: None) command = client_config._molmcp_command() @@ -111,52 +136,377 @@ def test_fallback_uses_this_interpreter_not_a_bare_python(self, monkeypatch): class TestOneJsonForEveryHost: - """Every host molmcp targets reads the standard `mcpServers` JSON. - - Grok loads ~/.claude.json, .cursor/mcp.json and project .mcp.json - alongside its own config.toml, and Claude Code and Cursor read the same - shape. Hand-rolling TOML bought nothing and cost an escaping bug, so - there is one body now and the host only picks where to put it. - """ - def test_the_body_is_identical_for_every_host(self): - toggle = client_config.PlaneToggle(("catalog",), (), ("catalog",)) + toggle = client_config.PlaneToggle(("molcrafts",), (), ("molcrafts",)) bodies = { - host: client_config.render_client(host, available=toggle.all_planes)[1] - for host in ("grok", "claude", "cursor") + host: client_config.render_init(host, available=toggle.all_planes)[1] + for host in ("grok", "claude", "cursor", "codex") } assert len(set(bodies.values())) == 1 - @pytest.mark.parametrize("host", ["grok", "claude", "cursor"]) + @pytest.mark.parametrize("host", ["grok", "claude", "cursor", "codex"]) def test_every_host_gets_parseable_json(self, host): - _, text = client_config.render_client(host) + _, text = client_config.render_init(host, available=("molcrafts",)) assert "mcpServers" in json.loads(text) - def test_the_host_is_optional(self): - _, text = client_config.render_client() - assert "mcpServers" in json.loads(text) +#: Production modules this file reads as text, so a deleted table stays deleted. +_SRC = Path(__file__).resolve().parents[1] / "src" / "molmcp" +CLIENT_CONFIG_SOURCE = _SRC / "client_config.py" +CLI_SOURCE = _SRC / "cli.py" - def test_disabled_planes_are_absent_rather_than_flagged(self): - toggle = client_config.PlaneToggle(("catalog",), ("molq",), ("catalog", "molq")) +#: Names ``client_config`` must hand back from ``molmcp.host`` unchanged. +RE_EXPORTED_NAMES: tuple[str, ...] = ( + "Host", + "SKILL_NAME", + "HOSTS", + "layout_for", + "default_write_path", +) - servers = client_config.render_mcp_json(toggle)["mcpServers"] +#: Names ``client_config`` re-exported while the host package was being split +#: out, and no longer does. They have one importable home, ``molmcp.host``: +#: reaching them through this module must fail rather than quietly work. +WITHDRAWN_NAMES: tuple[str, ...] = ( + "install_skill", + "skill_template", + "default_skill_dir", +) + +#: Every host ``molmcp init`` wires, in the order ``--help`` prints them. +INIT_HOSTS: tuple[str, ...] = ("grok", "claude", "cursor", "codex") + +#: The write primitives ``cli._init`` composes, in the order it must call them. +#: +#: ``install_harness_components`` is the activated-commit route — the pointer +#: one ``molmcp harness sync`` promoted, read down to the files its catalog +#: declares — and it is last for a reason that is not cosmetic. The placement +#: seam protects the managed usage skill by *skipping* a destination inside +#: that directory, which protects a file only once it is there, so the step +#: has to run after ``install_skill`` has written the constitution. Everything +#: between is the ``--source`` checkout route, which this one joins rather +#: than replaces. +INIT_PRIMITIVES: tuple[str, ...] = ( + "install_skill", + "install_extra_skills", + "write_adapter", + "install_harness_components", +) - assert set(servers) == {"catalog"} - @pytest.mark.parametrize( - ("host", "tail"), - [ - ("claude", ".claude.json"), - ("cursor", "mcp.json"), - ("grok", "mcp.json"), - ], +def _init_function() -> ast.FunctionDef: + """The ``cli._init`` definition, parsed from source rather than imported.""" + tree = ast.parse(CLI_SOURCE.read_text(encoding="utf-8")) + return next( + node + for node in ast.walk(tree) + if isinstance(node, ast.FunctionDef) and node.name == "_init" ) - def test_each_host_has_a_default_destination(self, host, tail): - assert str(client_config.default_write_path(host)).endswith(tail) - def test_no_toml_is_generated_any_more(self): - assert not hasattr(client_config, "render_grok_toml") + +def _calls_to(node: ast.AST, name: str) -> list[ast.Call]: + """Every bare-name call to *name* anywhere under *node*.""" + return [ + found + for found in ast.walk(node) + if isinstance(found, ast.Call) + and isinstance(found.func, ast.Name) + and found.func.id == name + ] + + +def _statement_indices(body: list[ast.stmt], name: str) -> list[int]: + """Positions of the top-level statements of *body* that call *name*.""" + return [index for index, stmt in enumerate(body) if _calls_to(stmt, name)] + + +def _args_source_reads(node: ast.AST) -> list[ast.Attribute]: + """Every ``args.source`` read under *node*.""" + return [ + found + for found in ast.walk(node) + if isinstance(found, ast.Attribute) + and found.attr == "source" + and isinstance(found.value, ast.Name) + and found.value.id == "args" + ] + + +def _resolved_binding(function: ast.FunctionDef) -> str: + """Name bound to the single ``resolve_bundle_source(...)`` result.""" + for stmt in function.body: + if isinstance(stmt, ast.Assign | ast.AnnAssign): + value = stmt.value + if ( + value is not None + and isinstance(value, ast.Call) + and isinstance(value.func, ast.Name) + and value.func.id == "resolve_bundle_source" + ): + target = ( + stmt.targets[0] if isinstance(stmt, ast.Assign) else stmt.target + ) + assert isinstance(target, ast.Name) + return target.id + raise AssertionError("_init binds no name to resolve_bundle_source(...)") + + +def _argument_names(call: ast.Call) -> set[str]: + """Bare names passed to *call*, positionally or by keyword.""" + passed = [*call.args, *(keyword.value for keyword in call.keywords)] + return {node.id for node in passed if isinstance(node, ast.Name)} + + +class TestHostTableLivesOnlyInTheHostPackage: + """One host path table: ``molmcp.host``. ``client_config`` only reads it.""" + + def test_the_private_host_dicts_are_gone_from_the_source(self) -> None: + text = CLIENT_CONFIG_SOURCE.read_text(encoding="utf-8") + + assert "_HOST_PATHS" not in text + assert "_HOST_SKILL_DIRS" not in text + + @pytest.mark.parametrize("name", RE_EXPORTED_NAMES) + def test_the_re_export_is_the_same_object_not_a_wrapper(self, name: str) -> None: + assert getattr(client_config, name) is getattr(host_package, name) + + @pytest.mark.parametrize("name", WITHDRAWN_NAMES) + def test_a_withdrawn_name_is_neither_attribute_nor_export(self, name: str) -> None: + assert not hasattr(client_config, name) + assert name not in client_config.__all__ + + def test_an_unknown_host_names_the_known_hosts_in_sorted_order(self) -> None: + with pytest.raises(ValueError) as excinfo: + render_init("nope") + + assert str(excinfo.value) == ( + "unknown host 'nope'; known: claude, codex, cursor, grok" + ) + + +class TestInitParserKeepsOneHostList: + """``--help`` still offers the same four hosts, derived from ``HOSTS``.""" + + def test_help_offers_the_four_hosts_in_declaration_order( + self, capsys: pytest.CaptureFixture[str] + ) -> None: + from molmcp import cli + + with pytest.raises(SystemExit): + cli.main(["init", "--help"]) + + assert "{" + ",".join(INIT_HOSTS) + "}" in capsys.readouterr().out + + def test_the_cli_repeats_no_second_host_list(self) -> None: + tree = ast.parse(CLI_SOURCE.read_text(encoding="utf-8")) + + host_literals = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.List | ast.Tuple | ast.Set) + and set(INIT_HOSTS) + <= { + element.value + for element in node.elts + if isinstance(element, ast.Constant) and isinstance(element.value, str) + } + ] + + assert host_literals == [] + + +class TestInitComposesTheHostPrimitives: + """``cli._init`` resolves the checkout once, then writes in a fixed order.""" + + def test_each_primitive_is_its_own_statement_in_order(self) -> None: + body = _init_function().body + + positions = {name: _statement_indices(body, name) for name in INIT_PRIMITIVES} + + assert {name: len(found) for name, found in positions.items()} == dict.fromkeys( + INIT_PRIMITIVES, 1 + ) + ordered = [positions[name][0] for name in INIT_PRIMITIVES] + assert ordered == sorted(set(ordered)) + + def test_catalog_components_are_placed_after_the_constitution_exists( + self, + ) -> None: + """The activated-commit route runs once ``install_skill`` has written. + + Stated on its own as well as through the tuple above, because it is + the one ordering constraint with a reason rather than a convention: + ``place_components`` keeps a catalog off the managed usage skill by + skipping any destination inside that directory, and skipping protects + a file that is already there. Placed before ``install_skill``, the + refusal would still fire and the constitution would then be written + over whatever the catalog had put in its place. + """ + body = _init_function().body + + assert ( + _statement_indices(body, "install_skill")[0] + < _statement_indices(body, "install_harness_components")[0] + ) + + def test_init_does_not_take_a_source_flag( + self, + tmp_path: Path, + monkeypatch: pytest.MonkeyPatch, + capsys: pytest.CaptureFixture[str], + ) -> None: + from molmcp import cli + + monkeypatch.setattr(Path, "home", lambda: tmp_path) + with pytest.raises(SystemExit) as ei: + cli.main(["init", "grok", "--source", str(tmp_path)]) + assert ei.value.code == 2 + assert "unrecognized arguments" in capsys.readouterr().err + + +#: The shipped usage constitution, read straight from the package it lives in. +#: The host package exposes no accessor for it — ``skill_template`` is in +#: :data:`WITHDRAWN_NAMES` above — and ``install_skill`` copies this very file. +SKILL_FILE = Path(skill_package.__file__).parent / "SKILL.md" + +#: The only install line that buys back a missing core. Frozen: nothing else +#: restores ``packages`` / ``open`` / ``route``. +CORE_INSTALL = "pip install molcrafts-molmcp" + +#: Plane -> the distribution its namespace needs, frozen by the provider +#: cutover. ``molexp`` publishes under its own name; the other two are +#: prefixed. A reader who follows one of these must land on a real project. +SCIENCE_PACKAGES: tuple[tuple[str, str], ...] = ( + ("molvis", "molcrafts-molvis"), + ("molq", "molcrafts-molq"), + ("molexp", "molexp"), +) + +#: A call the skill must never teach a model to make. ``require_upstream`` is +#: provider-internal, is reachable through no MCP tool, and recovers nothing. +FORBIDDEN_SKILL_CALL = "require_upstream" + +#: ``-mcp`` distribution names. None are published, so naming one +#: turns the recovery into a ``pip install`` that can only fail. +MCP_SUFFIXED_PACKAGE = re.compile(r"[\w-]+-mcp\b") + +#: Any pip line at all, used to prove where install advice is allowed to live. +PIP_INSTALL = re.compile(r"pip install ") + + +def _skill_text() -> str: + """The packaged ``SKILL.md``, as the agent that loads the skill reads it.""" + return SKILL_FILE.read_text(encoding="utf-8") + + +def _sections(text: str, marker: str) -> dict[str, str]: + """Body of every *marker*-level markdown heading, keyed by its title. + + A deeper heading stays inside its parent's body, so splitting on ``##`` + hands back whole sections and splitting one of those on ``###`` hands + back that section's numbered paths. + """ + prefix = f"{marker} " + found: dict[str, str] = {} + title = "" + body: list[str] = [] + for line in text.splitlines(): + if line.startswith(prefix): + if title: + found[title] = "\n".join(body) + title, body = line[len(prefix) :].strip(), [] + elif title: + body.append(line) + if title: + found[title] = "\n".join(body) + return found + + +def _recovery_paths() -> tuple[str, ...]: + """The numbered paths of the one section that recovers a missing tool.""" + text = _skill_text() + owning = [body for body in _sections(text, "##").values() if CORE_INSTALL in body] + assert len(owning) == 1, "the core install line must have exactly one home" + return tuple(_sections(owning[0], "###").values()) + + +class TestSkillOffersTwoRecoveriesAndNoThird: + """A missing tool has two causes, and the skill separates their fixes. + + The core being absent and a namespaced plane being absent look the same + to a model and need opposite answers, so the constitution splits them. + Both fixes end in a ``pip install``; each name below is pinned because a + wrong one sends the user to a project that does not exist. + """ + + def test_the_recovery_section_splits_into_exactly_two_paths(self) -> None: + assert len(_recovery_paths()) == 2 + + def test_the_first_path_installs_the_core(self) -> None: + first, _second = _recovery_paths() + + assert CORE_INSTALL in first + + def test_the_second_path_never_installs_the_core_again(self) -> None: + _first, second = _recovery_paths() + + assert CORE_INSTALL not in second + + def test_the_second_path_reopens_a_disabled_plane_before_installing( + self, + ) -> None: + _first, second = _recovery_paths() + + installs = [match.start() for match in PIP_INSTALL.finditer(second)] + + assert "--disable" in second + assert installs != [] + assert second.index("--disable") < min(installs) + + @pytest.mark.parametrize(("plane", "package"), SCIENCE_PACKAGES) + def test_a_plane_names_its_frozen_science_package( + self, plane: str, package: str + ) -> None: + _first, second = _recovery_paths() + exact = re.compile(rf"install\s+{re.escape(package)}(?![\w-])") + + rows = [ + line for line in second.splitlines() if plane in line and exact.search(line) + ] + + assert len(rows) == 1 + + def test_the_skill_never_tells_a_model_to_call_require_upstream(self) -> None: + assert FORBIDDEN_SKILL_CALL not in _skill_text() + + def test_the_constitution_points_at_molexp_plan(self) -> None: + assert "/molexp-plan" in _skill_text() + + def test_no_recovery_names_an_unpublished_mcp_suffixed_package(self) -> None: + assert MCP_SUFFIXED_PACKAGE.findall(_skill_text()) == [] + + def test_every_install_line_lives_inside_a_recovery_path(self) -> None: + whole = len(PIP_INSTALL.findall(_skill_text())) + inside = sum(len(PIP_INSTALL.findall(path)) for path in _recovery_paths()) + + assert whole > 0 + assert inside == whole + + +class TestTheInstalledSkillIsThePinnedFile: + """``molmcp init`` hands the agent the file the pins above are read from.""" + + def test_init_copies_the_constitution_byte_for_byte( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + monkeypatch.setattr(Path, "home", lambda: tmp_path) + + written = host_package.install_skill("grok") + + text = written.read_text(encoding="utf-8") + assert "when-to-use:" in text + assert "metadata:" not in text + assert "SYMBOL_NOT_FOUND" in text diff --git a/tests/test_components/__init__.py b/tests/test_components/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_components/test_activate.py b/tests/test_components/test_activate.py new file mode 100644 index 0000000..f051a7a --- /dev/null +++ b/tests/test_components/test_activate.py @@ -0,0 +1,464 @@ +"""Activation.bind / stage / promote / rollback — fake GitTransport.""" + +from __future__ import annotations + +import dataclasses +import inspect +import io +import json +import tarfile +from pathlib import Path + +import pytest + +import molmcp +import molmcp.components +from molmcp.components import activate as activate_module +from molmcp.components.activate import ( + Activation, + ActivationError, + ActivationVersionError, + IneligibleShaError, + NothingStagedError, + NothingToRollbackError, +) +from molmcp.components.catalog import CatalogError +from molmcp.components.store import ImmutableGitStore + +CAPABILITIES = frozenset({"provider-sdk", "harness-catalog"}) +SHA_A = "a" * 40 +SHA_B = "b" * 40 +_OWNER = "acme" +_REPO = "widgets" +CANONICAL_TOML = """\ +requires = ["provider-sdk", "harness-catalog"] + +[[component]] +kind = "skill" +name = "daily" +path = "skills/daily/SKILL.md" + +[[component]] +kind = "rule" +name = "safety" +path = "rules/safety.md" + +[[component]] +kind = "provider" +name = "molvis" +path = "providers/molvis/provider.py" +entrypoint = "molmcp.providers.molvis:MolvisProvider" + +[[component]] +kind = "overlay" +name = "molpy" +path = "overlays/molpy/overlay.py" +entrypoint = "molpy.overlay:MolpyOverlay" + +[[component]] +kind = "agent" +name = "reviewer" +path = "agents/reviewer/AGENT.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.daily", "rule.safety", "provider.molvis", "overlay.molpy"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.daily", "agent.reviewer", "rule.safety", "provider.molvis"] +""" +_ASSIGN_ERRORS = (AttributeError, dataclasses.FrozenInstanceError) + + +def _github_tarball(repo: str, sha: str) -> bytes: + """GitHub-style tar.gz whose inner directory is ``{repo}-{sha}/``.""" + prefix = f"{repo}-{sha}" + members = {f"{prefix}/harness.toml": CANONICAL_TOML} + buf = io.BytesIO() + with tarfile.open(fileobj=buf, mode="w:gz") as tar: + for name, content in members.items(): + data = content.encode("utf-8") + info = tarfile.TarInfo(name=name) + info.size = len(data) + tar.addfile(info, io.BytesIO(data)) + return buf.getvalue() + + +class _FakeGitTransport: + """``fetch_archive`` only; ``resolve_commit`` raises if called.""" + + def __init__(self, archives: dict[str, bytes]) -> None: + self._archives = archives + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + return self._archives[sha] + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + raise AssertionError("Activation tests must not call resolve_commit") + + +def _pointer(tmp_path: Path) -> Path: + return tmp_path / "activation.json" + + +def _new_store(tmp_path: Path, *shas: str) -> ImmutableGitStore: + keys = shas or (SHA_A,) + archives = {sha: _github_tarball(_REPO, sha) for sha in keys} + return ImmutableGitStore(tmp_path / "store", _FakeGitTransport(archives)) + + +def _published(tmp_path: Path, *shas: str) -> ImmutableGitStore: + keys = shas or (SHA_A,) + store = _new_store(tmp_path, *keys) + for sha in keys: + store.publish(sha, owner=_OWNER, repo=_REPO) + return store + + +def _bind( + tmp_path: Path, + store: ImmutableGitStore | None = None, + *, + path: Path | None = None, +) -> Activation: + if store is None: + store = _new_store(tmp_path) + if path is None: + path = _pointer(tmp_path) + return Activation.bind(path, store=store, supported_capabilities=CAPABILITIES) + + +def _write_pointer(path: Path, payload: dict[str, object]) -> None: + path.write_text(json.dumps(payload), encoding="utf-8") + + +def _read_pointer(path: Path) -> dict[str, object]: + payload = json.loads(path.read_text(encoding="utf-8")) + assert isinstance(payload, dict) + return payload + + +def _stub_load_harness_catalog(monkeypatch: pytest.MonkeyPatch, stub: object) -> None: + monkeypatch.setattr( + "molmcp.components.activate.load_harness_catalog", + stub, + ) + + +def _public_names(obj: object) -> set[str]: + return {name for name in dir(obj) if not name.startswith("_")} + + +class TestActivation: + def test_bind_signature_path_positional_collaborators_keyword_only(self): + params = inspect.signature(Activation.bind).parameters + assert list(params) == ["path", "store", "supported_capabilities"] + assert params["path"].kind in ( + inspect.Parameter.POSITIONAL_ONLY, + inspect.Parameter.POSITIONAL_OR_KEYWORD, + ) + assert params["store"].kind is inspect.Parameter.KEYWORD_ONLY + assert params["supported_capabilities"].kind is inspect.Parameter.KEYWORD_ONLY + assert params["path"].default is inspect.Parameter.empty + assert params["store"].default is inspect.Parameter.empty + assert params["supported_capabilities"].default is inspect.Parameter.empty + + def test_bind_store_none_raises_type_error(self, tmp_path): + with pytest.raises(TypeError): + Activation.bind( + _pointer(tmp_path), + store=None, # type: ignore[arg-type] + supported_capabilities=CAPABILITIES, + ) + + def test_bind_supported_capabilities_none_raises_type_error(self, tmp_path): + store = _new_store(tmp_path) + with pytest.raises(TypeError): + Activation.bind( + _pointer(tmp_path), + store=store, + supported_capabilities=None, # type: ignore[arg-type] + ) + + def test_constructs_without_arguments_raises_type_error(self): + with pytest.raises(TypeError): + Activation() # type: ignore[call-arg] + + def test_constructs_with_path_only_raises_type_error(self, tmp_path): + with pytest.raises(TypeError): + Activation(_pointer(tmp_path)) # type: ignore[call-arg] + + def test_bind_is_classmethod(self): + assert hasattr(Activation, "bind") + assert isinstance(inspect.getattr_static(Activation, "bind"), classmethod) + + def test_from_record_is_private(self): + assert hasattr(Activation, "_from_record") + assert Activation._from_record.__name__.startswith("_") + + def test_bind_missing_pointer_sets_current_previous_staged_none(self, tmp_path): + activation = _bind(tmp_path) + assert activation.current is None + assert activation.previous is None + assert activation.staged is None + + def test_bind_missing_pointer_does_not_create_path(self, tmp_path): + path = _pointer(tmp_path) + _bind(tmp_path, path=path) + assert not path.exists() + + def test_instance_has_no_public_active_attribute(self, tmp_path): + activation = _bind(tmp_path) + assert "active" not in _public_names(activation) + assert hasattr(activation, "current") + + def test_instance_has_no_public_staging_attribute(self, tmp_path): + activation = _bind(tmp_path) + assert "staging" not in _public_names(activation) + assert hasattr(activation, "staged") + + def test_assigning_current_raises(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(_ASSIGN_ERRORS): + activation.current = SHA_A # type: ignore[misc] + + def test_assigning_previous_raises(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(_ASSIGN_ERRORS): + activation.previous = SHA_A # type: ignore[misc] + + def test_assigning_staged_raises(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(_ASSIGN_ERRORS): + activation.staged = SHA_A # type: ignore[misc] + + def test_bind_unknown_json_version_raises_activation_version_error(self, tmp_path): + path = _pointer(tmp_path) + _write_pointer( + path, + {"version": 2, "active": None, "staging": None, "previous": None}, + ) + with pytest.raises(ActivationVersionError): + _bind(tmp_path, path=path) + + def test_bind_unknown_json_field_raises_activation_version_error(self, tmp_path): + path = _pointer(tmp_path) + _write_pointer( + path, + { + "version": 1, + "active": None, + "staging": None, + "previous": None, + "extra": True, + }, + ) + with pytest.raises(ActivationVersionError): + _bind(tmp_path, path=path) + + def test_bind_invalid_json_raises_activation_version_error(self, tmp_path): + path = _pointer(tmp_path) + path.write_text("{not-json", encoding="utf-8") + with pytest.raises(ActivationVersionError): + _bind(tmp_path, path=path) + + def test_activate_module_has_no_activation_unbound_error(self): + assert not hasattr(activate_module, "ActivationUnboundError") + assert "ActivationUnboundError" not in dir(activate_module) + + def test_stage_calls_load_harness_catalog_with_three_positional_args( + self, tmp_path, monkeypatch + ): + store = _published(tmp_path, SHA_A) + recorded: list[tuple[tuple[object, ...], dict[str, object]]] = [] + + def fake_load(*args: object, **kwargs: object) -> object: + recorded.append((args, kwargs)) + return object() + + _stub_load_harness_catalog(monkeypatch, fake_load) + activation = _bind(tmp_path, store) + activation.stage(SHA_A) + assert recorded == [ + ((store.tree_path(SHA_A), SHA_A, CAPABILITIES), {}), + ] + + def test_stage_sets_staged_without_changing_current_or_previous(self, tmp_path): + store = _published(tmp_path, SHA_A) + activation = _bind(tmp_path, store) + activation.stage(SHA_A) + assert activation.staged == SHA_A + assert activation.current is None + assert activation.previous is None + + def test_stage_writes_pointer_json_matching_properties(self, tmp_path): + store = _published(tmp_path, SHA_A) + path = _pointer(tmp_path) + activation = _bind(tmp_path, store, path=path) + activation.stage(SHA_A) + payload = _read_pointer(path) + assert payload == { + "version": 1, + "active": None, + "staging": SHA_A, + "previous": None, + } + assert activation.current == payload["active"] + assert activation.staged == payload["staging"] + assert activation.previous == payload["previous"] + + def test_stage_unpublished_sha_raises_ineligible_sha_error(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(IneligibleShaError): + activation.stage(SHA_A) + + def test_stage_unpublished_sha_does_not_create_pointer_file(self, tmp_path): + path = _pointer(tmp_path) + activation = _bind(tmp_path, path=path) + with pytest.raises(IneligibleShaError): + activation.stage(SHA_A) + assert not path.exists() + + def test_stage_unpublished_sha_does_not_mutate_existing_pointer(self, tmp_path): + path = _pointer(tmp_path) + original = { + "version": 1, + "active": None, + "staging": None, + "previous": None, + } + _write_pointer(path, original) + activation = _bind(tmp_path, path=path) + with pytest.raises(IneligibleShaError): + activation.stage(SHA_A) + assert _read_pointer(path) == original + + def test_stage_catalog_error_raises_ineligible_sha_error( + self, tmp_path, monkeypatch + ): + store = _published(tmp_path, SHA_A) + + def boom(*_args: object, **_kwargs: object) -> object: + raise CatalogError("ineligible") + + _stub_load_harness_catalog(monkeypatch, boom) + activation = _bind(tmp_path, store) + with pytest.raises(IneligibleShaError): + activation.stage(SHA_A) + + def test_ineligible_sha_error_subclasses_activation_error(self): + assert issubclass(IneligibleShaError, ActivationError) + + def test_promote_signature_has_only_self(self): + params = inspect.signature(Activation.promote).parameters + assert list(params) == ["self"] + assert "sha" not in params + + def test_rollback_signature_has_only_self(self): + params = inspect.signature(Activation.rollback).parameters + assert list(params) == ["self"] + assert "sha" not in params + + def test_promote_with_no_staged_raises_nothing_staged_error(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(NothingStagedError): + activation.promote() + + def test_rollback_with_no_previous_raises_nothing_to_rollback_error(self, tmp_path): + activation = _bind(tmp_path) + with pytest.raises(NothingToRollbackError): + activation.rollback() + + def test_promote_moves_staged_to_current_and_clears_staged(self, tmp_path): + store = _published(tmp_path, SHA_A) + path = _pointer(tmp_path) + activation = _bind(tmp_path, store, path=path) + activation.stage(SHA_A) + activation.promote() + assert activation.current == SHA_A + assert activation.staged is None + assert activation.previous is None + payload = _read_pointer(path) + assert payload == { + "version": 1, + "active": SHA_A, + "staging": None, + "previous": None, + } + assert activation.current == payload["active"] + assert activation.staged == payload["staging"] + assert activation.previous == payload["previous"] + + def test_second_promote_moves_current_to_previous(self, tmp_path): + store = _published(tmp_path, SHA_A, SHA_B) + path = _pointer(tmp_path) + activation = _bind(tmp_path, store, path=path) + activation.stage(SHA_A) + activation.promote() + activation.stage(SHA_B) + activation.promote() + assert activation.current == SHA_B + assert activation.staged is None + assert activation.previous == SHA_A + payload = _read_pointer(path) + assert payload == { + "version": 1, + "active": SHA_B, + "staging": None, + "previous": SHA_A, + } + assert activation.current == payload["active"] + assert activation.staged == payload["staging"] + assert activation.previous == payload["previous"] + + def test_rollback_restores_previous_and_clears_previous(self, tmp_path): + store = _published(tmp_path, SHA_A, SHA_B) + path = _pointer(tmp_path) + activation = _bind(tmp_path, store, path=path) + activation.stage(SHA_A) + activation.promote() + activation.stage(SHA_B) + activation.promote() + activation.rollback() + assert activation.current == SHA_A + assert activation.staged is None + assert activation.previous is None + payload = _read_pointer(path) + assert payload == { + "version": 1, + "active": SHA_A, + "staging": None, + "previous": None, + } + assert activation.current == payload["active"] + assert activation.staged == payload["staging"] + assert activation.previous == payload["previous"] + + def test_second_instance_promote_does_not_wipe_original_current(self, tmp_path): + store = _published(tmp_path, SHA_A, SHA_B) + path = _pointer(tmp_path) + first = _bind(tmp_path, store, path=path) + first.stage(SHA_A) + first.promote() + second = _bind(tmp_path, store, path=path) + assert second.current == SHA_A + second.stage(SHA_B) + second.promote() + assert second.current == SHA_B + assert second.previous == SHA_A + assert second.staged is None + + def test_activation_is_in_components_all(self): + assert "Activation" in molmcp.components.__all__ + + def test_immutable_git_store_is_in_components_all(self): + assert "ImmutableGitStore" in molmcp.components.__all__ + + def test_activation_is_not_in_molmcp_all(self): + assert "Activation" not in molmcp.__all__ + + def test_immutable_git_store_is_not_in_molmcp_all(self): + assert "ImmutableGitStore" not in molmcp.__all__ diff --git a/tests/test_components/test_catalog.py b/tests/test_components/test_catalog.py new file mode 100644 index 0000000..0915943 --- /dev/null +++ b/tests/test_components/test_catalog.py @@ -0,0 +1,879 @@ +"""HarnessCatalog construction, lookup, enable filtering, and harness.toml loading.""" + +from __future__ import annotations + +import ast +import dataclasses +import inspect +import sys +from pathlib import Path + +import pytest + +import molmcp +from molmcp.components.catalog import ( + HarnessCatalog, + ResolvedBundle, + load_harness_catalog, +) +from molmcp.components.models import ( + ALLOWED_REQUIRES, + BundleSpec, + CatalogError, + ComponentKind, + ComponentSpec, +) + +SHA = "0123456789abcdef0123456789abcdef01234567" +CAPABILITIES = frozenset({"provider-sdk", "harness-catalog"}) +CANONICAL_TOML = """\ +requires = ["provider-sdk", "harness-catalog"] + +[[component]] +kind = "skill" +name = "daily" +path = "skills/daily/SKILL.md" + +[[component]] +kind = "rule" +name = "safety" +path = "rules/safety.md" + +[[component]] +kind = "provider" +name = "molvis" +path = "providers/molvis/provider.py" +entrypoint = "molmcp.providers.molvis:MolvisProvider" + +[[component]] +kind = "overlay" +name = "molpy" +path = "overlays/molpy/overlay.py" +entrypoint = "molpy.overlay:MolpyOverlay" + +[[component]] +kind = "agent" +name = "reviewer" +path = "agents/reviewer/AGENT.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.daily", "rule.safety", "provider.molvis", "overlay.molpy"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.daily", "agent.reviewer", "rule.safety", "provider.molvis"] +""" +HARNESS_REPO_TOML = """\ +requires = ["provider-sdk", "harness-catalog"] +component_root = "plugins/mol" + +[[component]] +kind = "skill" +name = "spec" +path = "skills/spec/SKILL.md" + +[[component]] +kind = "agent" +name = "scientist" +path = "agents/scientist.md" + +[[component]] +kind = "rule" +name = "large-spec-split" +path = "rules/large-spec-split.md" + +[[component]] +kind = "provider" +name = "demo" +path = "providers/demo/provider.py" +entrypoint = "molmcp.providers.demo:DemoProvider" + +[[component]] +kind = "overlay" +name = "demo" +path = "overlays/demo/overlay.py" +entrypoint = "demo.overlay:DemoOverlay" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.spec", "rule.large-spec-split"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.spec", "agent.scientist", "rule.large-spec-split"] +""" +"""A catalog spelled the way ``MolCrafts/harness`` authors its own rows. + +This is deliberately **not** "the same shape" as the real file. The real +repository is 55 rows and ships **zero** providers and **zero** overlays; +this fixture carries three rows in its real kind census -- a skill at +``skills//SKILL.md``, an agent at ``agents/.md``, a rule at +``rules/.md`` -- plus one provider and one overlay row that exist +only to cover the loader's kind table. Nothing here builds a fold or +asserts anything about either arm. +""" + +#: Every value ``_validate_component_root`` must refuse. Each is refused +#: twice over: through ``load_harness_catalog`` and through direct +#: ``HarnessCatalog(component_root=...)`` construction, because the value +#: gate lives in ``__post_init__`` and only the loader can see presence. +_REJECTED_COMPONENT_ROOTS = ( + "..", + "../evil", + "a/../b", + ".", + "/plugins", + "plugins\\mol", + "D:evil", +) +#: Values that must load. ``plugins/mol`` is two segments, so the path +#: separator check ``ImmutableGitStore._sha_dir`` and ``harness.pointer_path`` +#: both carry is deliberately absent from this guard. +_ACCEPTED_COMPONENT_ROOTS = ("plugins/mol", "plugins/mol/nested") +_COMPONENTS_DIR = Path(__file__).resolve().parents[2] / "src" / "molmcp" / "components" +_DAILY_IDS = ( + "skill.daily", + "rule.safety", + "provider.molvis", + "overlay.molpy", +) +_DEV_IDS = ( + "skill.daily", + "agent.reviewer", + "rule.safety", + "provider.molvis", +) + + +def _write_harness_toml(tmp_path: Path, content: str = CANONICAL_TOML) -> Path: + path = tmp_path / "harness.toml" + path.write_text(content, encoding="utf-8") + return path + + +def _toml_with_component_root(value: str, base: str = CANONICAL_TOML) -> str: + r"""Return ``base`` with ``component_root = value`` above its first table. + + The key is written as a TOML *literal* string (single quotes) so that + ``plugins\mol`` reaches the guard as data. In a basic string a lone + backslash is an invalid escape, and the loader would answer with a + ``TOMLDecodeError`` wrapped as ``invalid harness.toml`` before + ``_validate_component_root`` ever saw the value. + + A bare key after a ``[[component]]`` header is itself a + ``TOMLDecodeError``, so the key goes beside ``requires`` at the top. + """ + + return f"component_root = '{value}'\n{base}" + + +def _catalog_with_root(component_root: str) -> HarnessCatalog: + """Construct a catalog directly, passing ``component_root`` by keyword. + + Bypasses the loader on purpose: the *value* gate lives in + ``HarnessCatalog.__post_init__``, so a bad value must be + unconstructible even when no ``harness.toml`` exists. + """ + + return HarnessCatalog( + sha=SHA, + requires=("provider-sdk", "harness-catalog"), + components=_leaf_components(), + bundles=(_daily_bundle(), _dev_bundle()), + component_root=component_root, + ) + + +def _leaf_components() -> tuple[ComponentSpec, ...]: + return ( + ComponentSpec( + kind=ComponentKind.SKILL, + name="daily", + id="skill.daily", + path="skills/daily/SKILL.md", + ), + ComponentSpec( + kind=ComponentKind.RULE, + name="safety", + id="rule.safety", + path="rules/safety.md", + ), + ComponentSpec( + kind=ComponentKind.PROVIDER, + name="molvis", + id="provider.molvis", + path="providers/molvis/provider.py", + entrypoint="molmcp.providers.molvis:MolvisProvider", + ), + ComponentSpec( + kind=ComponentKind.OVERLAY, + name="molpy", + id="overlay.molpy", + path="overlays/molpy/overlay.py", + entrypoint="molpy.overlay:MolpyOverlay", + ), + ComponentSpec( + kind=ComponentKind.AGENT, + name="reviewer", + id="agent.reviewer", + path="agents/reviewer/AGENT.md", + ), + ) + + +def _daily_bundle( + *, + requires: tuple[str, ...] = (), + members: tuple[str, ...] = _DAILY_IDS, +) -> BundleSpec: + return BundleSpec(name="daily", members=members, requires=requires) + + +def _dev_bundle( + *, + requires: tuple[str, ...] = (), + members: tuple[str, ...] = _DEV_IDS, +) -> BundleSpec: + return BundleSpec(name="dev", members=members, requires=requires) + + +def _enable_leaves() -> tuple[ComponentSpec, ...]: + return ( + ComponentSpec( + kind=ComponentKind.SKILL, + name="notes", + id="skill.notes", + path="skills/notes/SKILL.md", + ), + ComponentSpec( + kind=ComponentKind.RULE, + name="style", + id="rule.style", + path="rules/style.md", + ), + ComponentSpec( + kind=ComponentKind.AGENT, + name="reviewer", + id="agent.reviewer", + path="agents/reviewer/AGENT.md", + ), + ) + + +def _sci_bundle() -> BundleSpec: + return BundleSpec(name="sci", members=("skill.notes", "rule.style")) + + +def _lab_bundle() -> BundleSpec: + return BundleSpec(name="lab", members=("skill.notes", "agent.reviewer")) + + +def _enable_catalog( + *, + bundles: tuple[BundleSpec, ...] | None = None, +) -> HarnessCatalog: + return _catalog( + components=_enable_leaves(), + bundles=(_sci_bundle(), _lab_bundle()) if bundles is None else bundles, + ) + + +def _catalog( + *, + sha: str = SHA, + requires: tuple[str, ...] = ("provider-sdk", "harness-catalog"), + components: tuple[ComponentSpec, ...] | None = None, + bundles: tuple[BundleSpec, ...] | None = None, +) -> HarnessCatalog: + return HarnessCatalog( + sha=sha, + requires=requires, + components=_leaf_components() if components is None else components, + bundles=((_daily_bundle(), _dev_bundle()) if bundles is None else bundles), + ) + + +def _member_ids(resolved: ResolvedBundle) -> tuple[str, ...]: + return tuple(member.id for member in resolved.members) + + +def _toml_without_bundle(name: str) -> str: + marker = f'name = "{name}"' + chunks: list[str] = [] + skipping = False + for raw in CANONICAL_TOML.split("[[component]]"): + if not skipping and marker in raw and "members" in raw: + skipping = True + continue + chunks.append(raw) + return "[[component]]".join(chunks) + + +def _non_stdlib_imports(path: Path) -> list[str]: + tree = ast.parse(path.read_text(encoding="utf-8")) + found: list[str] = [] + for node in ast.walk(tree): + names: list[str] = [] + if isinstance(node, ast.Import): + names = [alias.name for alias in node.names] + elif isinstance(node, ast.ImportFrom): + if node.level: + continue + if node.module: + names = [node.module] + for name in names: + top = name.split(".")[0] + if top in sys.stdlib_module_names or top == "__future__": + continue + if name == "molmcp.components" or name.startswith("molmcp.components."): + continue + found.append(name) + return found + + +class TestHarnessCatalog: + def test_is_frozen(self): + catalog = _catalog() + with pytest.raises(dataclasses.FrozenInstanceError): + catalog.sha = "0" * 40 # type: ignore[misc] + + def test_uses_slots(self): + catalog = _catalog() + assert not hasattr(catalog, "__dict__") + assert hasattr(HarnessCatalog, "__slots__") + + def test_constructs_with_valid_sha(self): + catalog = _catalog() + assert catalog.sha == SHA + assert catalog.requires == ("provider-sdk", "harness-catalog") + + @pytest.mark.parametrize( + "sha", + [ + "not-a-sha", + "0123456789ABCDEF0123456789ABCDEF01234567", + "0123456789abcdef0123456789abcdef0123456", + "0123456789abcdef0123456789abcdef012345678", + ], + ) + def test_rejects_invalid_sha(self, sha): + with pytest.raises(CatalogError): + _catalog(sha=sha) + + def test_rejects_unknown_requires_token(self): + assert "not-a-capability" not in ALLOWED_REQUIRES + with pytest.raises(CatalogError): + _catalog(requires=("not-a-capability",)) + + def test_constructs_with_only_sci_bundle(self): + catalog = _enable_catalog(bundles=(_sci_bundle(),)) + assert tuple(bundle.name for bundle in catalog.bundles) == ("sci",) + + def test_constructs_with_empty_bundles(self): + catalog = _enable_catalog(bundles=()) + assert catalog.bundles == () + assert tuple(spec.id for spec in catalog.components) == ( + "skill.notes", + "rule.style", + "agent.reviewer", + ) + + def test_rejects_duplicate_component_ids(self): + leaves = _leaf_components() + with pytest.raises(CatalogError): + _catalog(components=leaves + (leaves[0],)) + + def test_rejects_duplicate_bundle_names(self): + with pytest.raises(CatalogError): + _catalog(bundles=(_daily_bundle(), _daily_bundle(), _dev_bundle())) + + def test_rejects_unknown_bundle_member_id(self): + with pytest.raises(CatalogError): + _catalog( + bundles=( + _daily_bundle(members=("skill.daily", "skill.missing")), + _dev_bundle(), + ) + ) + + def test_constructs_by_keyword_without_component_root(self): + catalog = _catalog() + assert catalog.component_root == "" + + def test_component_root_is_declared_last(self): + names = tuple(field.name for field in dataclasses.fields(HarnessCatalog)) + assert names[-1] == "component_root" + + @pytest.mark.parametrize("component_root", _ACCEPTED_COMPONENT_ROOTS) + def test_accepts_multi_segment_component_root(self, component_root): + """``plugins/mol`` is two segments and must stay legal. + + ``ImmutableGitStore._sha_dir`` and ``harness.pointer_path`` both + refuse path separators, because a SHA and a source name are + interpolated as single segments. ``component_root`` is the opposite + case, so that half of the borrowed guard is deliberately absent. + Assert it, or a later "simplification" restores the separator check + and breaks the only layout this key exists to support. + """ + + assert _catalog_with_root(component_root).component_root == component_root + + def test_empty_component_root_is_constructible(self): + """``""`` means "the tree itself" and must construct. + + At construction time a defaulted ``""`` and a written ``""`` are the + same string, so ``__post_init__`` cannot tell them apart and must not + try. The empty-when-present refusal belongs to the loader, which is + the only gate that can still see presence. + """ + + assert _catalog_with_root("").component_root == "" + + @pytest.mark.parametrize("component_root", _REJECTED_COMPONENT_ROOTS) + def test_rejects_escaping_component_root(self, component_root): + with pytest.raises(CatalogError) as ei: + _catalog_with_root(component_root) + assert repr(component_root) in str(ei.value) + + def test_rejects_slash_prefixed_component_root_that_is_not_absolute(self): + """``/plugins`` needs the second clause of the absolute-path test. + + ``PureWindowsPath("/plugins").is_absolute()`` is ``False``, so + ``Path(value).is_absolute()`` alone misses it off Windows -- while + ``PureWindowsPath("C:/store/tree") / "/plugins"`` is + ``WindowsPath("C:/plugins")``, the base gone. The guard needs + ``or value.startswith("/")``, exactly the pair + ``_validate_component_path`` already carries. + """ + + assert "/plugins" in _REJECTED_COMPONENT_ROOTS + with pytest.raises(CatalogError) as ei: + _catalog_with_root("/plugins") + assert repr("/plugins") in str(ei.value) + + def test_rejects_drive_relative_component_root(self): + """``D:evil`` passes every other clause and still discards the base. + + It carries no ``..``, holds no backslash, and + ``Path("D:evil").is_absolute()`` is ``False`` on POSIX -- yet + ``PureWindowsPath("C:/store/tree") / "D:evil"`` is + ``WindowsPath("D:evil")``: a drive on the *first* joined component + resets the anchor and drops the base entirely. ``component_root`` is + always that first component, and CI runs ``windows-latest``. + """ + + assert "D:evil" in _REJECTED_COMPONENT_ROOTS + with pytest.raises(CatalogError) as ei: + _catalog_with_root("D:evil") + assert repr("D:evil") in str(ei.value) + + def test_rejects_dot_component_root_by_the_segment_predicate(self): + """``.`` is refused for its segment, not for being empty-when-present. + + It passes every other clause and ``Path("/store/tree") / "."`` is + ``/store/tree``, a second spelling of ``""``. The predicate splits on + ``"/"`` rather than reading ``PurePath.parts``, which silently drops + ``.`` and collapses ``//`` and would therefore miss it. This test + reaches the value gate directly, with no ``harness.toml`` anywhere, + so the loader's presence check cannot be what answers -- while + ``component_root=""`` on the same path constructs. + """ + + with pytest.raises(CatalogError) as ei: + _catalog_with_root(".") + assert repr(".") in str(ei.value) + + def test_has_no_supported_capabilities_field(self): + catalog = _catalog() + assert not hasattr(catalog, "supported_capabilities") + + def test_get_skill_daily_returns_component_spec(self): + spec = _catalog().get("skill.daily") + assert isinstance(spec, ComponentSpec) + assert spec.kind is ComponentKind.SKILL + assert spec.id == "skill.daily" + + def test_get_daily_raises_unknown_id(self): + with pytest.raises(CatalogError) as ei: + _catalog().get("daily") + assert "unknown-id" in str(ei.value) + + def test_get_missing_component_id_raises_unknown_id(self): + with pytest.raises(CatalogError) as ei: + _catalog().get("skill.missing") + assert "unknown-id" in str(ei.value) + + def test_get_bundle_daily_returns_bundle_spec(self): + spec = _catalog().get_bundle("daily") + assert isinstance(spec, BundleSpec) + assert spec.name == "daily" + assert spec.members == _DAILY_IDS + + def test_get_bundle_missing_raises_unknown_bundle(self): + with pytest.raises(CatalogError) as ei: + _catalog().get_bundle("missing") + assert "unknown-bundle" in str(ei.value) + + def test_resolve_bundle_daily_member_ids_are_golden(self): + resolved = _catalog().resolve_bundle("daily") + assert isinstance(resolved, ResolvedBundle) + assert _member_ids(resolved) == _DAILY_IDS + + def test_resolve_bundle_dev_member_ids_are_golden(self): + resolved = _catalog().resolve_bundle("dev") + assert isinstance(resolved, ResolvedBundle) + assert _member_ids(resolved) == _DEV_IDS + + def test_resolve_bundle_requires_is_first_seen_union(self): + catalog = _catalog( + requires=("provider-sdk",), + bundles=( + _daily_bundle(requires=("harness-catalog", "provider-sdk")), + _dev_bundle(), + ), + ) + resolved = catalog.resolve_bundle("daily") + assert resolved.requires == ("provider-sdk", "harness-catalog") + + def test_resolved_bundle_union_is_not_a_catalog_field(self): + catalog_fields = tuple( + field.name for field in dataclasses.fields(HarnessCatalog) + ) + assert catalog_fields == ( + "sha", + "requires", + "components", + "bundles", + "component_root", + ) + catalog = _catalog() + assert not hasattr(catalog, "resolved_requires") + bundle_fields = tuple(field.name for field in dataclasses.fields(BundleSpec)) + assert bundle_fields == ("name", "members", "requires") + resolved_fields = tuple( + field.name for field in dataclasses.fields(ResolvedBundle) + ) + assert resolved_fields == ("name", "members", "requires") + + def test_enabled_components_unions_sci_then_lab_in_first_seen_order(self): + catalog = _enable_catalog() + ids = tuple(spec.id for spec in catalog.enabled_components(("sci", "lab"))) + assert ids == ("skill.notes", "rule.style", "agent.reviewer") + + def test_enabled_components_unions_lab_then_sci_in_first_seen_order(self): + catalog = _enable_catalog() + ids = tuple(spec.id for spec in catalog.enabled_components(("lab", "sci"))) + assert ids == ("skill.notes", "agent.reviewer", "rule.style") + + def test_enabled_components_empty_tuple_returns_empty(self): + catalog = _enable_catalog() + assert catalog.enabled_components(()) == () + + def test_enabled_components_unknown_name_raises_unknown_bundle(self): + catalog = _enable_catalog() + with pytest.raises(CatalogError) as ei: + catalog.enabled_components(("nope",)) + message = str(ei.value) + assert "unknown-bundle" in message + assert "sci" in message + assert "lab" in message + + def test_enabled_components_none_with_empty_bundles_returns_all_components(self): + catalog = _enable_catalog(bundles=()) + assert catalog.components != () + assert catalog.enabled_components(None) == catalog.components + + def test_enabled_components_named_bundle_with_empty_bundles_raises(self): + catalog = _enable_catalog(bundles=()) + with pytest.raises(CatalogError) as ei: + catalog.enabled_components(("sci",)) + assert "unknown-bundle" in str(ei.value) + + +class TestLoadHarnessCatalog: + def test_load_stores_sha_argument(self, tmp_path): + _write_harness_toml(tmp_path) + sha = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" + catalog = load_harness_catalog(tmp_path, sha, CAPABILITIES) + assert catalog.sha == sha + + def test_two_argument_call_raises_type_error(self, tmp_path): + _write_harness_toml(tmp_path) + with pytest.raises(TypeError): + load_harness_catalog(tmp_path, SHA) # type: ignore[call-arg] + + def test_supported_capabilities_has_no_default(self): + param = inspect.signature(load_harness_catalog).parameters[ + "supported_capabilities" + ] + assert param.default is inspect.Parameter.empty + + def test_wire_bundle_rows_become_bundle_spec(self, tmp_path): + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert all(isinstance(bundle, BundleSpec) for bundle in catalog.bundles) + assert {bundle.name for bundle in catalog.bundles} == {"daily", "dev"} + assert "bundle.daily" not in {spec.id for spec in catalog.components} + assert all(isinstance(spec, ComponentSpec) for spec in catalog.components) + + def test_five_kind_rows_become_component_spec(self, tmp_path): + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert all(isinstance(spec, ComponentSpec) for spec in catalog.components) + assert {spec.kind for spec in catalog.components} == { + ComponentKind.SKILL, + ComponentKind.AGENT, + ComponentKind.RULE, + ComponentKind.PROVIDER, + ComponentKind.OVERLAY, + } + + def test_get_daily_is_unknown_id_after_load(self, tmp_path): + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + with pytest.raises(CatalogError) as ei: + catalog.get("daily") + assert "unknown-id" in str(ei.value) + + def test_get_bundle_daily_works_after_load(self, tmp_path): + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + spec = catalog.get_bundle("daily") + assert isinstance(spec, BundleSpec) + assert spec.members == _DAILY_IDS + + def test_component_kind_still_has_no_bundle(self): + assert not hasattr(ComponentKind, "BUNDLE") + assert "bundle" not in {member.value for member in ComponentKind} + with pytest.raises(ValueError) as ei: + ComponentKind("bundle") + assert type(ei.value) is ValueError + + def test_empty_capabilities_is_ineligible_when_requires_present(self, tmp_path): + _write_harness_toml(tmp_path) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, frozenset()) + assert "ineligible" in str(ei.value) + + def test_missing_harness_catalog_capability_is_ineligible(self, tmp_path): + _write_harness_toml(tmp_path) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, frozenset({"provider-sdk"})) + assert "ineligible" in str(ei.value) + + def test_bundle_requires_missing_from_capabilities_is_ineligible(self, tmp_path): + content = CANONICAL_TOML.replace( + 'requires = ["provider-sdk", "harness-catalog"]', + 'requires = ["provider-sdk"]', + ).replace( + 'name = "daily"\nmembers = ["skill.daily", "rule.safety", ' + '"provider.molvis", "overlay.molpy"]', + 'name = "daily"\nmembers = ["skill.daily", "rule.safety", ' + '"provider.molvis", "overlay.molpy"]\nrequires = ["harness-catalog"]', + ) + _write_harness_toml(tmp_path, content) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, frozenset({"provider-sdk"})) + assert "ineligible" in str(ei.value) + + def test_unknown_requires_token_fails_language_gate_before_eligibility( + self, tmp_path + ): + content = CANONICAL_TOML.replace( + 'requires = ["provider-sdk", "harness-catalog"]', + 'requires = ["not-a-capability"]', + ) + _write_harness_toml(tmp_path, content) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, frozenset({"not-a-capability"})) + assert "ineligible" not in str(ei.value) + + def test_loads_component_root_from_the_file(self, tmp_path): + _write_harness_toml(tmp_path, _toml_with_component_root("plugins/mol")) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.component_root == "plugins/mol" + + def test_absent_component_root_is_the_empty_string(self, tmp_path): + """The canonical file names no ``component_root`` and still loads. + + Paired with ``test_rejects_empty_component_root_when_present``: only + the difference between the two proves a presence check exists at all. + """ + + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.component_root == "" + + def test_rejects_empty_component_root_when_present(self, tmp_path): + """``component_root = ""`` is refused, though an absent key is not. + + The loader is the only gate that can still see presence: + ``__post_init__`` receives ``""`` from a defaulted field and from a + written one alike. + + The refusal must not be ``_reject_unknown``'s: an unrecognised key + already raises ``CatalogError`` naming ``component_root``, so + without the second assertion this test passes before the key exists. + """ + + _write_harness_toml(tmp_path, _toml_with_component_root("")) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert "component_root" in str(ei.value) + assert "unknown field" not in str(ei.value) + + def test_component_root_is_not_an_unknown_field(self, tmp_path): + """Asserted behaviourally, never against ``catalog._TOP_LEVEL_KEYS``. + + Reading that constant back would be true the instant an implementer + edits the line, which is not evidence that the key parses. + """ + + _write_harness_toml(tmp_path, _toml_with_component_root("plugins/mol")) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert isinstance(catalog, HarnessCatalog) + + def test_component_paths_are_carried_not_rewritten(self, tmp_path): + """With a ``component_root`` set, ``path`` comes out exactly as authored. + + The expected literal is written out here rather than derived from the + TOML input, so the assertion can fail. ``component_root`` is carried + beside the paths and never folded into them. + """ + + _write_harness_toml(tmp_path, _toml_with_component_root("plugins/mol")) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.get("skill.daily").path == "skills/daily/SKILL.md" + assert catalog.get("rule.safety").path == "rules/safety.md" + assert catalog.get("provider.molvis").path == "providers/molvis/provider.py" + assert catalog.get("overlay.molpy").path == "overlays/molpy/overlay.py" + assert catalog.get("agent.reviewer").path == "agents/reviewer/AGENT.md" + + def test_component_root_does_not_weaken_the_kind_prefix(self, tmp_path): + """A rooted catalog still refuses a path missing its kind prefix. + + ``KIND_PATH_PREFIX`` validates paths unchanged; ``component_root`` is + not a licence to drop ``skills/`` from a skill row. + """ + + content = _toml_with_component_root( + "plugins/mol", + CANONICAL_TOML.replace( + 'path = "skills/daily/SKILL.md"', 'path = "daily/SKILL.md"' + ), + ) + _write_harness_toml(tmp_path, content) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert "skills/" in str(ei.value) + + @pytest.mark.parametrize("component_root", _ACCEPTED_COMPONENT_ROOTS) + def test_loads_multi_segment_component_root(self, tmp_path, component_root): + _write_harness_toml(tmp_path, _toml_with_component_root(component_root)) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.component_root == component_root + + @pytest.mark.parametrize("component_root", _REJECTED_COMPONENT_ROOTS) + def test_rejects_escaping_component_root(self, tmp_path, component_root): + _write_harness_toml(tmp_path, _toml_with_component_root(component_root)) + with pytest.raises(CatalogError) as ei: + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert repr(component_root) in str(ei.value) + + def test_harness_repo_shaped_catalog_loads(self, tmp_path): + """The real repository's authored spellings, through the real loader. + + No ``_wire`` seam and no monkeypatch: this is the only evidence in + this repository that the ``MolCrafts/harness`` layout parses at all. + """ + + _write_harness_toml(tmp_path, HARNESS_REPO_TOML) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.component_root == "plugins/mol" + assert catalog.get("skill.spec").path == "skills/spec/SKILL.md" + assert catalog.get("agent.scientist").path == "agents/scientist.md" + assert catalog.get("rule.large-spec-split").path == "rules/large-spec-split.md" + + def test_harness_repo_shaped_catalog_covers_the_kind_table(self, tmp_path): + _write_harness_toml(tmp_path, HARNESS_REPO_TOML) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert {spec.kind for spec in catalog.components} == set(ComponentKind) + assert {bundle.name for bundle in catalog.bundles} == {"daily", "dev"} + + def test_rejects_unknown_top_level_key(self, tmp_path): + _write_harness_toml(tmp_path, CANONICAL_TOML + "\nunexpected = 1\n") + with pytest.raises(CatalogError): + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + + @pytest.mark.parametrize("field", ["sha", "version", "tag", "release", "id"]) + def test_rejects_identity_top_level_field(self, tmp_path, field): + _write_harness_toml(tmp_path, CANONICAL_TOML + f'\n{field} = "1"\n') + with pytest.raises(CatalogError): + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + + def test_rejects_unknown_component_kind(self, tmp_path): + extra = """ +[[component]] +kind = "widget" +name = "extra" +path = "widgets/extra.md" +""" + _write_harness_toml(tmp_path, CANONICAL_TOML + extra) + with pytest.raises(CatalogError): + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + + def test_loads_without_daily_bundle(self, tmp_path): + _write_harness_toml(tmp_path, _toml_without_bundle("daily")) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert tuple(bundle.name for bundle in catalog.bundles) == ("dev",) + + def test_loads_without_dev_bundle(self, tmp_path): + _write_harness_toml(tmp_path, _toml_without_bundle("dev")) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert tuple(bundle.name for bundle in catalog.bundles) == ("daily",) + + def test_does_not_load_catalog_toml(self, tmp_path): + (tmp_path / "catalog.toml").write_text(CANONICAL_TOML, encoding="utf-8") + with pytest.raises(CatalogError): + load_harness_catalog(tmp_path, SHA, CAPABILITIES) + + def test_does_not_import_entrypoint_module(self, tmp_path): + sys.modules.pop("does.not.exist", None) + content = CANONICAL_TOML.replace( + "molmcp.providers.molvis:MolvisProvider", + "does.not.exist:Nope", + ) + _write_harness_toml(tmp_path, content) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.get("provider.molvis").entrypoint == "does.not.exist:Nope" + assert "does.not.exist" not in sys.modules + + def test_does_not_require_component_paths_on_disk(self, tmp_path): + _write_harness_toml(tmp_path) + catalog = load_harness_catalog(tmp_path, SHA, CAPABILITIES) + assert catalog.get("skill.daily").id == "skill.daily" + for spec in catalog.components: + assert not (tmp_path / spec.path).exists() + + def test_components_package_imports_only_stdlib(self): + sources = [ + _COMPONENTS_DIR / "catalog.py", + _COMPONENTS_DIR / "models.py", + _COMPONENTS_DIR / "__init__.py", + ] + for path in sources: + assert path.is_file(), f"missing {path.name}" + imported = _non_stdlib_imports(path) + assert imported == [], f"{path.name} imports {imported}" + + def test_symbols_are_not_in_molmcp_all(self): + exported = set(molmcp.__all__) + assert "load_harness_catalog" not in exported + assert "HarnessCatalog" not in exported + assert "ComponentSpec" not in exported + assert "ComponentKind" not in exported diff --git a/tests/test_components/test_git.py b/tests/test_components/test_git.py new file mode 100644 index 0000000..3e6b499 --- /dev/null +++ b/tests/test_components/test_git.py @@ -0,0 +1,605 @@ +"""The two GitTransport implementations and extract_git_archive. + +``GitHubTransport`` is driven against a fake ``urlopen``: no socket is +opened here. ``LocalGitTransport`` is the opposite kind of leaf — it shells +out to ``git`` against a checkout this module builds in ``tmp_path``, so it +is driven against a *real* repository rather than a mock. Neither reaches +the network, and no ``DiscoveryEngine`` is involved in either. +""" + +from __future__ import annotations + +import inspect +import io +import json +import subprocess +import tarfile +import urllib.error +import urllib.request +from email.message import Message +from pathlib import Path +from typing import NamedTuple + +import pytest + +from molmcp.components import git as git_mod +from molmcp.components.git import ( + GitError, + GitHubTransport, + GitTransport, + extract_git_archive, +) + +_OWNER = "owner" +_REPO = "repo" +_SHA = "a" * 40 +_API = "https://api.github.com" +_CODELOAD = "https://codeload.github.com" +_REPO_URL = f"{_API}/repos/{_OWNER}/{_REPO}" +_COMMITS_DEV = f"{_API}/repos/{_OWNER}/{_REPO}/commits/dev" +_COMMITS_MAIN = f"{_API}/repos/{_OWNER}/{_REPO}/commits/main" +_ARCHIVE_URL = f"{_CODELOAD}/{_OWNER}/{_REPO}/tar.gz/{_SHA}" +_GIT_PY = ( + Path(__file__).resolve().parents[2] / "src" / "molmcp" / "components" / "git.py" +) + + +def _json_body(payload: dict[str, object]) -> bytes: + return json.dumps(payload).encode("utf-8") + + +def _headers(request: urllib.request.Request) -> dict[str, str]: + return {key.lower(): value for key, value in request.header_items()} + + +def _make_tarball(members: dict[str, str]) -> bytes: + buf = io.BytesIO() + with tarfile.open(fileobj=buf, mode="w:gz") as tar: + for name, content in members.items(): + data = content.encode("utf-8") + info = tarfile.TarInfo(name=name) + info.size = len(data) + tar.addfile(info, io.BytesIO(data)) + return buf.getvalue() + + +class _FakeResponse: + def __init__(self, body: bytes) -> None: + self._body = body + + def read(self) -> bytes: + return self._body + + def __enter__(self) -> _FakeResponse: + return self + + def __exit__(self, *exc: object) -> None: + return None + + +class _FakeUrlOpen: + """Stand-in for ``urllib.request.urlopen``; never opens a socket.""" + + def __init__( + self, + body_for: dict[str, bytes] | None = None, + *, + error: BaseException | None = None, + ) -> None: + self.body_for = body_for or {} + self.error = error + self.calls: list[tuple[urllib.request.Request, float | None]] = [] + + def __call__( + self, + request: urllib.request.Request, + timeout: float | None = None, + ) -> _FakeResponse: + self.calls.append((request, timeout)) + if self.error is not None: + raise self.error + url = request.full_url + if url not in self.body_for: + raise AssertionError(f"unexpected urlopen url: {url}") + return _FakeResponse(self.body_for[url]) + + +def _install(monkeypatch: pytest.MonkeyPatch, fake: _FakeUrlOpen) -> _FakeUrlOpen: + monkeypatch.setattr(urllib.request, "urlopen", fake) + return fake + + +class TestGitHubTransport: + def test_protocol_declares_resolve_commit_and_fetch_archive(self): + assert callable(getattr(GitTransport, "resolve_commit", None)) + assert callable(getattr(GitTransport, "fetch_archive", None)) + + def test_git_error_is_runtime_error(self): + assert issubclass(GitError, RuntimeError) + + def test_constructs_without_arguments(self): + GitHubTransport() + + def test_constructs_with_token_none(self): + GitHubTransport(token=None) + + def test_constructor_takes_only_token(self): + params = inspect.signature(GitHubTransport).parameters + assert list(params) == ["token"] + assert params["token"].default is None + with pytest.raises(TypeError): + GitHubTransport(timeout=30) # type: ignore[call-arg] + with pytest.raises(TypeError): + GitHubTransport(config=None) # type: ignore[call-arg] + + def test_resolve_commit_with_ref_hits_commits_url_and_returns_sha( + self, monkeypatch + ): + fake = _install( + monkeypatch, + _FakeUrlOpen({_COMMITS_DEV: _json_body({"sha": _SHA})}), + ) + sha = GitHubTransport().resolve_commit(_OWNER, _REPO, ref="dev") + assert sha == _SHA + assert len(fake.calls) == 1 + request, _timeout = fake.calls[0] + assert request.full_url == _COMMITS_DEV + + def test_resolve_commit_without_ref_resolves_default_branch_first( + self, monkeypatch + ): + fake = _install( + monkeypatch, + _FakeUrlOpen( + { + _REPO_URL: _json_body({"default_branch": "main"}), + _COMMITS_MAIN: _json_body({"sha": _SHA}), + } + ), + ) + sha = GitHubTransport().resolve_commit(_OWNER, _REPO, ref=None) + assert sha == _SHA + assert [request.full_url for request, _timeout in fake.calls] == [ + _REPO_URL, + _COMMITS_MAIN, + ] + + def test_fetch_archive_hits_codeload_and_returns_bytes(self, monkeypatch): + payload = b"tarball-bytes" + fake = _install(monkeypatch, _FakeUrlOpen({_ARCHIVE_URL: payload})) + data = GitHubTransport().fetch_archive(_OWNER, _REPO, _SHA) + assert data == payload + assert len(fake.calls) == 1 + request, _timeout = fake.calls[0] + assert request.full_url == _ARCHIVE_URL + + def test_user_agent_is_exactly_molmcp(self, monkeypatch): + fake = _install( + monkeypatch, + _FakeUrlOpen( + { + _REPO_URL: _json_body({"default_branch": "main"}), + _COMMITS_MAIN: _json_body({"sha": _SHA}), + _ARCHIVE_URL: b"tarball-bytes", + } + ), + ) + transport = GitHubTransport() + transport.resolve_commit(_OWNER, _REPO, ref=None) + transport.fetch_archive(_OWNER, _REPO, _SHA) + assert fake.calls, "expected urlopen to be called" + for request, _timeout in fake.calls: + assert _headers(request)["user-agent"] == "molmcp" + + def test_token_sends_authorization_bearer(self, monkeypatch): + fake = _install( + monkeypatch, + _FakeUrlOpen({_COMMITS_DEV: _json_body({"sha": _SHA})}), + ) + GitHubTransport(token="test-token").resolve_commit(_OWNER, _REPO, ref="dev") + request, _timeout = fake.calls[0] + assert _headers(request)["authorization"] == "Bearer test-token" + + def test_token_none_omits_authorization(self, monkeypatch): + fake = _install( + monkeypatch, + _FakeUrlOpen({_COMMITS_DEV: _json_body({"sha": _SHA})}), + ) + GitHubTransport(token=None).resolve_commit(_OWNER, _REPO, ref="dev") + request, _timeout = fake.calls[0] + assert "authorization" not in _headers(request) + + def test_urlopen_timeout_is_thirty_seconds(self, monkeypatch): + fake = _install( + monkeypatch, + _FakeUrlOpen({_COMMITS_DEV: _json_body({"sha": _SHA})}), + ) + GitHubTransport().resolve_commit(_OWNER, _REPO, ref="dev") + _request, timeout = fake.calls[0] + assert timeout == 30 + + @pytest.mark.parametrize("code", [404, 503]) + def test_http_error_raises_git_error(self, monkeypatch, code): + error = urllib.error.HTTPError(_COMMITS_DEV, code, "error", Message(), None) + _install(monkeypatch, _FakeUrlOpen(error=error)) + with pytest.raises(GitError): + GitHubTransport().resolve_commit(_OWNER, _REPO, ref="dev") + + def test_url_error_raises_git_error(self, monkeypatch): + _install( + monkeypatch, + _FakeUrlOpen(error=urllib.error.URLError("connection refused")), + ) + with pytest.raises(GitError): + GitHubTransport().resolve_commit(_OWNER, _REPO, ref="dev") + + def test_json_payload_without_sha_raises_git_error(self, monkeypatch): + _install( + monkeypatch, + _FakeUrlOpen({_COMMITS_DEV: _json_body({"message": "ok"})}), + ) + with pytest.raises(GitError): + GitHubTransport().resolve_commit(_OWNER, _REPO, ref="dev") + + def test_source_does_not_read_the_environment(self): + text = _GIT_PY.read_text(encoding="utf-8") + assert "os.environ" not in text + assert "getenv" not in text + + +class TestExtractGitArchive: + def test_extracts_inner_root_and_file(self, tmp_path): + dest = tmp_path / "raw" + dest.mkdir() + data = _make_tarball({"repo-sha/calc.py": "x = 1"}) + root = extract_git_archive(data, dest) + assert root == dest / "repo-sha" + inner = dest / "repo-sha" / "calc.py" + assert inner.is_file() + assert inner.read_text(encoding="utf-8") == "x = 1" + + def test_empty_bytes_raises_git_error(self, tmp_path): + dest = tmp_path / "raw" + dest.mkdir() + with pytest.raises(GitError): + extract_git_archive(b"", dest) + + def test_corrupt_bytes_raises_git_error(self, tmp_path): + dest = tmp_path / "raw" + dest.mkdir() + with pytest.raises(GitError): + extract_git_archive(b"this is not a tar.gz", dest) + + def test_tarball_with_no_directory_entry_raises_git_error(self, tmp_path): + dest = tmp_path / "raw" + dest.mkdir() + data = _make_tarball({"calc.py": "x = 1"}) + with pytest.raises(GitError): + extract_git_archive(data, dest) + + +_BRANCH = "dev" +_TAG = "v1" +_ANNOTATED_TAG = "v1-signed-off" +_MANIFEST = '[harness]\nname = "mine"\n' +_SKILL = "# greet\n" +_SCRATCH = "still being edited\n" +_IDENTITY = ( + "-c", + "user.name=molmcp tests", + "-c", + "user.email=tests@molmcp.invalid", +) + + +class _Checkout(NamedTuple): + """A real git repository built under ``tmp_path``. + + Two commits, so a ref that is not ``HEAD`` has somewhere else to point: + ``tagged`` carries only ``harness.toml`` and is what ``dev``, the + lightweight ``v1`` and the annotated ``v1-signed-off`` all name; + ``head`` adds ``skills/greet.md`` on ``main``. One more file — + ``scratch.txt`` — sits in the working tree, committed by nothing. + """ + + root: Path + head: str + tagged: str + + +def _git(root: Path, *args: str) -> str: + """Run one git command inside ``root`` and return its stripped stdout.""" + result = subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + text=True, + ) + return result.stdout.strip() + + +def _init(root: Path) -> None: + """Create ``root`` as an empty repository on ``main`` with 40-hex SHAs.""" + root.mkdir(parents=True, exist_ok=True) + _git(root, "init", "-q", "--initial-branch=main", "--object-format=sha1") + + +def _commit(root: Path, message: str) -> str: + """Commit everything currently in ``root`` and return the new SHA.""" + _git(root, "add", "-A") + _git(root, *_IDENTITY, "commit", "--no-gpg-sign", "-q", "-m", message) + return _git(root, "rev-parse", "HEAD") + + +def _write(path: Path, text: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def _tree(root: Path) -> set[str]: + """Every file under ``root``, as slash-separated relative paths.""" + return { + item.relative_to(root).as_posix() for item in root.rglob("*") if item.is_file() + } + + +def _extract(data: bytes, tmp_path: Path, name: str) -> Path: + """Extract ``data`` into a fresh directory and return the inner tree.""" + dest = tmp_path / name + dest.mkdir() + return extract_git_archive(data, dest) + + +@pytest.fixture +def checkout(tmp_path: Path) -> _Checkout: + root = tmp_path / "harness" + _init(root) + _write(root / "harness.toml", _MANIFEST) + tagged = _commit(root, "first") + _git(root, "tag", _TAG) + _git( + root, + *_IDENTITY, + "-c", + "tag.gpgSign=false", + "tag", + "-a", + _ANNOTATED_TAG, + "-m", + "release one", + ) + _git(root, "branch", _BRANCH) + _write(root / "skills" / "greet.md", _SKILL) + head = _commit(root, "second") + _write(root / "scratch.txt", _SCRATCH) + return _Checkout(root=root, head=head, tagged=tagged) + + +class TestLocalGitTransport: + """A harness source that is a checkout on disk rather than a coordinate. + + Same two primitives as :class:`GitHubTransport` — resolve a ref to a + commit SHA, hand back that commit's gzip tarball — read out of a local + repository instead of over HTTP. ``owner`` and ``repo`` are accepted + because the ``GitTransport`` protocol passes them, and are *ignored*: + the ``root`` this was constructed with is the whole repository + selection, which is the one difference worth pinning. + + The property that makes a local source a *source* rather than a + directory read is that ``fetch_archive`` archives the committed tree at + a SHA — never the working tree. Without it, "pinned to a commit" would + mean "whatever the operator had unsaved at the moment we looked", and + there would be no reason to go through git at all instead of copying + the directory. + + The class is reached through ``git_mod`` rather than imported by name + at module scope on purpose: while it does not exist, every test here + fails on its own ``AttributeError`` instead of one collection error + taking :class:`TestGitHubTransport` down with it. + """ + + def test_constructor_takes_only_root(self) -> None: + params = inspect.signature(git_mod.LocalGitTransport).parameters + assert list(params) == ["root"] + + def test_the_methods_take_the_protocol_parameters(self) -> None: + for method in ("resolve_commit", "fetch_archive"): + assert list( + inspect.signature(getattr(git_mod.LocalGitTransport, method)).parameters + ) == list(inspect.signature(getattr(GitTransport, method)).parameters) + + def test_resolve_commit_of_head_returns_a_forty_hex_sha( + self, checkout: _Checkout + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, "HEAD" + ) + + assert sha == checkout.head + assert len(sha) == 40 + assert set(sha) <= set("0123456789abcdef") + + def test_resolve_commit_of_the_checked_out_branch_is_the_head_commit( + self, checkout: _Checkout + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, "main" + ) + + assert sha == checkout.head + + def test_resolve_commit_of_another_branch_is_that_branchs_tip( + self, checkout: _Checkout + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, _BRANCH + ) + + assert sha == checkout.tagged + assert sha != checkout.head + + def test_resolve_commit_of_a_tag_is_the_tagged_commit( + self, checkout: _Checkout + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, _TAG + ) + + assert sha == checkout.tagged + + def test_resolve_commit_of_an_annotated_tag_is_the_commit_not_the_tag_object( + self, checkout: _Checkout + ) -> None: + """``git rev-parse`` on an annotated tag yields the *tag object*. + + The protocol promises a commit SHA, and a tag object's SHA is not + one — an activation pinned to it would name something ``git log`` + cannot walk. ``git tag -a`` is how a harness release gets cut, so + this is the ordinary case rather than an exotic one. + """ + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, _ANNOTATED_TAG + ) + + assert sha == checkout.tagged + + def test_resolve_commit_of_a_sha_is_that_same_sha( + self, checkout: _Checkout + ) -> None: + transport = git_mod.LocalGitTransport(checkout.root) + + assert transport.resolve_commit(_OWNER, _REPO, checkout.tagged) == ( + checkout.tagged + ) + + def test_resolve_commit_without_a_ref_takes_the_default_branch( + self, checkout: _Checkout + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, None + ) + + assert sha == checkout.head + + @pytest.mark.parametrize( + ("owner", "repo"), + [("", ""), ("acme", "somewhere-else"), ("MolCrafts", "harness")], + ) + def test_owner_and_repo_do_not_select_the_repository( + self, checkout: _Checkout, owner: str, repo: str + ) -> None: + sha = git_mod.LocalGitTransport(checkout.root).resolve_commit( + owner, repo, "HEAD" + ) + + assert sha == checkout.head + + def test_the_root_is_what_selects_the_repository( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + other = tmp_path / "other" + _init(other) + _write(other / "harness.toml", '[harness]\nname = "other"\n') + other_head = _commit(other, "only") + + assert other_head != checkout.head + assert ( + git_mod.LocalGitTransport(checkout.root).resolve_commit(_OWNER, _REPO, None) + == checkout.head + ) + assert ( + git_mod.LocalGitTransport(other).resolve_commit(_OWNER, _REPO, None) + == other_head + ) + + def test_fetch_archive_returns_bytes_extract_git_archive_accepts( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + data = git_mod.LocalGitTransport(checkout.root).fetch_archive( + _OWNER, _REPO, checkout.head + ) + + assert isinstance(data, bytes) + inner = _extract(data, tmp_path, "raw") + assert inner.is_dir() + + def test_the_archived_tree_holds_the_committed_files( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + data = git_mod.LocalGitTransport(checkout.root).fetch_archive( + _OWNER, _REPO, checkout.head + ) + + inner = _extract(data, tmp_path, "raw") + assert _tree(inner) == {"harness.toml", "skills/greet.md"} + assert (inner / "harness.toml").read_text(encoding="utf-8") == _MANIFEST + + def test_the_archive_is_the_committed_tree_not_the_working_tree( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + assert (checkout.root / "scratch.txt").is_file(), "fixture wrote no scratch" + + data = git_mod.LocalGitTransport(checkout.root).fetch_archive( + _OWNER, _REPO, checkout.head + ) + + inner = _extract(data, tmp_path, "raw") + assert "scratch.txt" not in _tree(inner) + + def test_an_earlier_sha_archives_that_commits_tree( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + data = git_mod.LocalGitTransport(checkout.root).fetch_archive( + _OWNER, _REPO, checkout.tagged + ) + + inner = _extract(data, tmp_path, "raw") + assert _tree(inner) == {"harness.toml"} + + def test_the_inner_directory_names_the_commit_it_was_taken_at( + self, tmp_path: Path, checkout: _Checkout + ) -> None: + data = git_mod.LocalGitTransport(checkout.root).fetch_archive( + _OWNER, _REPO, checkout.head + ) + + inner = _extract(data, tmp_path, "raw") + assert checkout.head in inner.name + + def test_an_unknown_ref_raises_git_error(self, checkout: _Checkout) -> None: + with pytest.raises(GitError) as excinfo: + git_mod.LocalGitTransport(checkout.root).resolve_commit( + _OWNER, _REPO, "no-such" + ) + + assert not isinstance(excinfo.value, subprocess.CalledProcessError) + + def test_an_unknown_sha_raises_git_error(self, checkout: _Checkout) -> None: + with pytest.raises(GitError) as excinfo: + git_mod.LocalGitTransport(checkout.root).fetch_archive(_OWNER, _REPO, _SHA) + + assert not isinstance(excinfo.value, subprocess.CalledProcessError) + + def test_a_root_that_is_not_a_repository_raises_git_error( + self, tmp_path: Path + ) -> None: + plain = tmp_path / "plain" + _write(plain / "harness.toml", _MANIFEST) + + with pytest.raises(GitError): + git_mod.LocalGitTransport(plain).resolve_commit(_OWNER, _REPO, "HEAD") + + def test_fetching_from_a_root_that_is_not_a_repository_raises_git_error( + self, tmp_path: Path + ) -> None: + plain = tmp_path / "plain" + _write(plain / "harness.toml", _MANIFEST) + + with pytest.raises(GitError): + git_mod.LocalGitTransport(plain).fetch_archive(_OWNER, _REPO, _SHA) + + def test_a_root_that_does_not_exist_raises_git_error(self, tmp_path: Path) -> None: + with pytest.raises(GitError): + git_mod.LocalGitTransport(tmp_path / "missing").resolve_commit( + _OWNER, _REPO, "HEAD" + ) diff --git a/tests/test_components/test_locator.py b/tests/test_components/test_locator.py new file mode 100644 index 0000000..db1163c --- /dev/null +++ b/tests/test_components/test_locator.py @@ -0,0 +1,249 @@ +"""parse_harness_locator — GitHub and local locators to one origin key.""" + +from __future__ import annotations + +import ast +import dataclasses +import sys +from pathlib import Path + +import pytest + +from molmcp.components.locator import ( + LocatorError, + ParsedHarnessLocator, + parse_harness_locator, +) + +_LOCATOR_PY = ( + Path(__file__).resolve().parents[2] / "src" / "molmcp" / "components" / "locator.py" +) +_SRC = Path(__file__).resolve().parents[2] / "src" / "molmcp" +_FORBIDDEN_LAYERS = ("molmcp.discovery", "molmcp.settings") +_FORBIDDEN_LIBS = ("urllib", "git") + + +def _imported_targets(path: Path) -> tuple[str, ...]: + """Absolute dotted import targets, with relative imports resolved.""" + package = ".".join(("molmcp", *path.relative_to(_SRC).parent.parts)) + parts = package.split(".") + found: list[str] = [] + for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))): + if isinstance(node, ast.Import): + found.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + base = parts[: len(parts) - (node.level - 1)] + tail = node.module.split(".") if node.module else [] + module = ".".join([*base, *tail]) + else: + module = node.module or "" + found.append(module) + found.extend(f"{module}.{alias.name}" for alias in node.names) + return tuple(found) + + +def _reaches(targets: tuple[str, ...], dotted: str) -> bool: + return any( + target == dotted or target.startswith(f"{dotted}.") for target in targets + ) + + +def _locator_source() -> Path: + if not _LOCATOR_PY.is_file(): + pytest.skip("src/molmcp/components/locator.py is not present") + return _LOCATOR_PY + + +class TestParseHarnessLocator: + def test_locator_error_is_a_value_error(self): + assert issubclass(LocatorError, ValueError) + + def test_parsed_locator_fields_are_the_six_documented_ones(self): + assert {field.name for field in dataclasses.fields(ParsedHarnessLocator)} == { + "locator", + "kind", + "origin_key", + "ref", + "owner", + "repo", + } + + def test_owner_repo_lowercases_origin_key(self): + parsed = parse_harness_locator("MolCrafts/harness") + assert parsed.locator == "MolCrafts/harness" + assert parsed.kind == "github" + assert parsed.origin_key == "molcrafts/harness" + assert parsed.owner == "molcrafts" + assert parsed.repo == "harness" + assert parsed.ref == "" + + def test_https_git_url_with_trailing_slash_shares_origin_key(self): + raw = "https://github.com/MolCrafts/harness.git/" + parsed = parse_harness_locator(raw) + assert parsed.locator == raw + assert parsed.kind == "github" + assert parsed.origin_key == "molcrafts/harness" + assert parsed.owner == "molcrafts" + assert parsed.repo == "harness" + assert parsed.ref == "" + + def test_host_prefixed_owner_repo_shares_origin_key(self): + parsed = parse_harness_locator("github.com/MolCrafts/harness") + assert parsed.origin_key == "molcrafts/harness" + assert parsed.kind == "github" + assert parsed.owner == "molcrafts" + assert parsed.repo == "harness" + assert parsed.ref == "" + + def test_already_lowercase_owner_repo_shares_origin_key(self): + parsed = parse_harness_locator("molcrafts/harness") + assert parsed.origin_key == "molcrafts/harness" + assert parsed.kind == "github" + assert parsed.owner == "molcrafts" + assert parsed.repo == "harness" + assert parsed.ref == "" + + def test_www_host_shares_origin_key(self): + parsed = parse_harness_locator("www.github.com/MolCrafts/harness") + assert parsed.origin_key == "molcrafts/harness" + assert parsed.kind == "github" + + def test_at_ref_is_not_part_of_origin_key(self): + parsed = parse_harness_locator("Owner/repo@dev") + assert parsed.locator == "Owner/repo@dev" + assert parsed.kind == "github" + assert parsed.origin_key == "owner/repo" + assert parsed.owner == "owner" + assert parsed.repo == "repo" + assert parsed.ref == "dev" + + def test_a_leading_slash_is_local_on_every_platform(self): + from pathlib import PureWindowsPath + + raw = "/opt/harness/mine" + # Windows Path.is_absolute() is False without a drive letter; a + # leading slash must still be a filesystem path, not owner/repo. + assert not PureWindowsPath(raw).is_absolute() + parsed = parse_harness_locator(raw) + assert parsed.kind == "local" + assert parsed.locator == raw + assert parsed.origin_key == str(Path(raw).expanduser().resolve()) + assert parsed.owner == "" + assert parsed.repo == "" + + def test_absolute_path_is_local_with_resolved_origin_key(self, tmp_path: Path): + raw = str(tmp_path / "harness") + parsed = parse_harness_locator(raw) + assert parsed.kind == "local" + assert parsed.origin_key == str(Path(raw).expanduser().resolve()) + assert parsed.locator == raw + assert parsed.ref == "" + assert parsed.owner == "" + assert parsed.repo == "" + + def test_home_relative_path_is_local_with_resolved_origin_key( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ): + home = tmp_path / "home" + home.mkdir() + monkeypatch.setattr(Path, "home", staticmethod(lambda: home)) + monkeypatch.setenv("HOME", str(home)) + monkeypatch.setenv("USERPROFILE", str(home)) + raw = "~/harness" + parsed = parse_harness_locator(raw) + assert parsed.kind == "local" + assert parsed.origin_key == str(Path(raw).expanduser().resolve()) + assert parsed.locator == raw + assert parsed.ref == "" + assert parsed.owner == "" + assert parsed.repo == "" + + def test_parsed_locator_is_frozen(self): + parsed = parse_harness_locator("molcrafts/harness") + with pytest.raises(dataclasses.FrozenInstanceError): + parsed.origin_key = "other" # type: ignore[misc] + + @pytest.mark.parametrize("raw", ["./checkout", "../checkout"]) + def test_relative_path_raises(self, raw: str): + with pytest.raises(LocatorError): + parse_harness_locator(raw) + + def test_empty_string_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator("") + + @pytest.mark.parametrize( + "raw", + [ + " MolCrafts/harness", + "MolCrafts/harness ", + "\tMolCrafts/harness", + "MolCrafts/harness\n", + "molcrafts / harness", + ], + ) + def test_whitespace_raises(self, raw: str): + with pytest.raises(LocatorError): + parse_harness_locator(raw) + + def test_http_url_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator("http://github.com/MolCrafts/harness") + + def test_github_scheme_prefix_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator("github:owner/repo") + + def test_url_with_extra_path_segment_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator("https://github.com/MolCrafts/harness/tree/main") + + def test_backslash_in_a_github_locator_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator(r"MolCrafts\harness") + + def test_a_windows_drive_path_is_local_only_on_windows(self): + raw = r"C:\harness" + if sys.platform == "win32": + parsed = parse_harness_locator(raw) + assert parsed.kind == "local" + assert parsed.locator == raw + assert parsed.origin_key == str(Path(raw).expanduser().resolve()) + else: + with pytest.raises(LocatorError): + parse_harness_locator(raw) + + def test_ssh_locator_raises(self): + with pytest.raises(LocatorError): + parse_harness_locator("git@github.com:MolCrafts/harness.git") + + @pytest.mark.parametrize("dotted", _FORBIDDEN_LAYERS) + def test_module_does_not_import_discovery_or_settings(self, dotted: str): + imported = _imported_targets(_locator_source()) + assert not _reaches(imported, dotted) + + @pytest.mark.parametrize("dotted", _FORBIDDEN_LIBS) + def test_module_does_not_import_urllib_or_git(self, dotted: str): + imported = _imported_targets(_locator_source()) + assert not _reaches(imported, dotted) + + def test_module_source_does_not_name_harness_source(self): + source = _locator_source().read_text(encoding="utf-8") + assert "HarnessSource" not in source + + def test_module_performs_no_import_the_walk_cannot_see(self): + tree = ast.parse(_locator_source().read_text(encoding="utf-8")) + dynamic = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Call) + and ( + (isinstance(node.func, ast.Name) and node.func.id == "import_module") + or ( + isinstance(node.func, ast.Attribute) + and node.func.attr in {"import_module", "__import__"} + ) + ) + ] + assert dynamic == [] diff --git a/tests/test_components/test_models.py b/tests/test_components/test_models.py new file mode 100644 index 0000000..3770fd4 --- /dev/null +++ b/tests/test_components/test_models.py @@ -0,0 +1,340 @@ +"""Leaf grammar for ComponentKind, ComponentSpec, and BundleSpec.""" + +from __future__ import annotations + +import dataclasses +import re +from enum import StrEnum + +import pytest + +from molmcp.components.models import ( + ALLOWED_REQUIRES, + COMPONENT_NAME_PATTERN, + KIND_PATH_PREFIX, + SHA_PATTERN, + BundleSpec, + CatalogError, + ComponentKind, + ComponentSpec, +) + + +def _pattern_source(value: str | re.Pattern[str]) -> str: + return value if isinstance(value, str) else value.pattern + + +def _skill_spec( + *, + kind: ComponentKind = ComponentKind.SKILL, + name: str = "daily", + id: str = "skill.daily", + path: str = "skills/daily/SKILL.md", + entrypoint: str | None = None, +) -> ComponentSpec: + return ComponentSpec(kind=kind, name=name, id=id, path=path, entrypoint=entrypoint) + + +def _agent_spec( + *, + entrypoint: str | None = None, +) -> ComponentSpec: + return ComponentSpec( + kind=ComponentKind.AGENT, + name="reviewer", + id="agent.reviewer", + path="agents/reviewer/AGENT.md", + entrypoint=entrypoint, + ) + + +def _rule_spec( + *, + entrypoint: str | None = None, +) -> ComponentSpec: + return ComponentSpec( + kind=ComponentKind.RULE, + name="safety", + id="rule.safety", + path="rules/safety.md", + entrypoint=entrypoint, + ) + + +def _provider_spec( + *, + kind: ComponentKind = ComponentKind.PROVIDER, + name: str = "molvis", + id: str = "provider.molvis", + path: str = "providers/molvis/provider.py", + entrypoint: str | None = "molmcp.providers.molvis:MolvisProvider", +) -> ComponentSpec: + return ComponentSpec(kind=kind, name=name, id=id, path=path, entrypoint=entrypoint) + + +def _overlay_spec( + *, + entrypoint: str | None = "molpy.overlay:MolpyOverlay", +) -> ComponentSpec: + return ComponentSpec( + kind=ComponentKind.OVERLAY, + name="molpy", + id="overlay.molpy", + path="overlays/molpy/overlay.py", + entrypoint=entrypoint, + ) + + +def _bundle_spec( + *, + name: str = "daily", + members: tuple[str, ...] = ("skill.daily", "rule.safety"), + requires: tuple[str, ...] = (), +) -> BundleSpec: + return BundleSpec(name=name, members=members, requires=requires) + + +class TestComponentKind: + def test_is_str_enum(self): + assert issubclass(ComponentKind, StrEnum) + + def test_members_are_exactly_the_five_leaf_kinds(self): + assert list(ComponentKind) == [ + ComponentKind.SKILL, + ComponentKind.AGENT, + ComponentKind.RULE, + ComponentKind.PROVIDER, + ComponentKind.OVERLAY, + ] + + def test_values_are_lowercase_strings(self): + assert ComponentKind.SKILL == "skill" + assert ComponentKind.AGENT == "agent" + assert ComponentKind.RULE == "rule" + assert ComponentKind.PROVIDER == "provider" + assert ComponentKind.OVERLAY == "overlay" + + def test_has_no_bundle_member(self): + assert not hasattr(ComponentKind, "BUNDLE") + assert "bundle" not in {member.value for member in ComponentKind} + + def test_constructing_bundle_raises_value_error(self): + with pytest.raises(ValueError) as ei: + ComponentKind("bundle") + assert type(ei.value) is ValueError + + def test_catalog_error_subclasses_value_error(self): + assert issubclass(CatalogError, ValueError) + + def test_sha_pattern_is_forty_lowercase_hex(self): + assert _pattern_source(SHA_PATTERN) == r"^[0-9a-f]{40}$" + + +class TestComponentSpec: + def test_component_name_pattern_is_the_kebab_regex(self): + assert _pattern_source(COMPONENT_NAME_PATTERN) == r"^[a-z][a-z0-9-]*$" + + def test_kind_path_prefix_maps_each_leaf_kind(self): + assert KIND_PATH_PREFIX[ComponentKind.SKILL] == "skills/" + assert KIND_PATH_PREFIX[ComponentKind.AGENT] == "agents/" + assert KIND_PATH_PREFIX[ComponentKind.RULE] == "rules/" + assert KIND_PATH_PREFIX[ComponentKind.PROVIDER] == "providers/" + assert KIND_PATH_PREFIX[ComponentKind.OVERLAY] == "overlays/" + + def test_constructs_skill_daily(self): + spec = ComponentSpec( + kind=ComponentKind.SKILL, + name="daily", + id="skill.daily", + path="skills/daily/SKILL.md", + ) + assert spec.kind is ComponentKind.SKILL + assert spec.name == "daily" + assert spec.id == "skill.daily" + assert spec.path == "skills/daily/SKILL.md" + assert spec.entrypoint is None + + def test_constructs_agent_reviewer(self): + spec = ComponentSpec( + kind=ComponentKind.AGENT, + name="reviewer", + id="agent.reviewer", + path="agents/reviewer/AGENT.md", + ) + assert spec.kind is ComponentKind.AGENT + assert spec.path == "agents/reviewer/AGENT.md" + assert spec.entrypoint is None + + def test_constructs_rule_safety(self): + spec = ComponentSpec( + kind=ComponentKind.RULE, + name="safety", + id="rule.safety", + path="rules/safety.md", + ) + assert spec.kind is ComponentKind.RULE + assert spec.path == "rules/safety.md" + assert spec.entrypoint is None + + def test_constructs_provider_molvis(self): + spec = ComponentSpec( + kind=ComponentKind.PROVIDER, + name="molvis", + id="provider.molvis", + path="providers/molvis/provider.py", + entrypoint="molmcp.providers.molvis:MolvisProvider", + ) + assert spec.kind is ComponentKind.PROVIDER + assert spec.path == "providers/molvis/provider.py" + assert spec.entrypoint == "molmcp.providers.molvis:MolvisProvider" + + def test_constructs_overlay_molpy(self): + spec = ComponentSpec( + kind=ComponentKind.OVERLAY, + name="molpy", + id="overlay.molpy", + path="overlays/molpy/overlay.py", + entrypoint="molpy.overlay:MolpyOverlay", + ) + assert spec.kind is ComponentKind.OVERLAY + assert spec.path == "overlays/molpy/overlay.py" + assert spec.entrypoint == "molpy.overlay:MolpyOverlay" + + def test_rejects_kind_that_is_not_component_kind(self): + with pytest.raises(CatalogError): + ComponentSpec( + kind="skill", # type: ignore[arg-type] + name="daily", + id="skill.daily", + path="skills/daily/SKILL.md", + ) + + def test_rejects_name_with_underscore(self): + with pytest.raises(CatalogError): + _skill_spec(name="daily_skill", id="skill.daily_skill") + + def test_rejects_uppercase_name(self): + with pytest.raises(CatalogError): + _skill_spec(name="Daily", id="skill.Daily") + + def test_rejects_empty_name(self): + with pytest.raises(CatalogError): + _skill_spec(name="", id="skill.") + + def test_rejects_id_mismatch(self): + with pytest.raises(CatalogError): + _skill_spec(name="daily", id="skill.other") + + def test_rejects_absolute_path(self): + with pytest.raises(CatalogError): + _skill_spec(path="/skills/daily/SKILL.md") + + def test_rejects_backslash_in_path(self): + with pytest.raises(CatalogError): + _skill_spec(path="skills\\daily\\SKILL.md") + + def test_rejects_dotdot_segment(self): + with pytest.raises(CatalogError): + _skill_spec(path="skills/../secret") + + def test_rejects_empty_path(self): + with pytest.raises(CatalogError): + _skill_spec(path="") + + def test_rejects_wrong_kind_prefix(self): + with pytest.raises(CatalogError): + _skill_spec(path="docs/daily.md") + + def test_rejects_prefix_with_nothing_after(self): + with pytest.raises(CatalogError): + _skill_spec(path="skills/") + + def test_rejects_provider_missing_entrypoint(self): + with pytest.raises(CatalogError): + _provider_spec(entrypoint=None) + + def test_rejects_provider_empty_entrypoint(self): + with pytest.raises(CatalogError): + _provider_spec(entrypoint="") + + def test_rejects_provider_entrypoint_without_colon(self): + with pytest.raises(CatalogError): + _provider_spec(entrypoint="molmcp.providers.molvis") + + def test_rejects_overlay_missing_entrypoint(self): + with pytest.raises(CatalogError): + _overlay_spec(entrypoint=None) + + def test_rejects_skill_with_entrypoint(self): + with pytest.raises(CatalogError): + _skill_spec(entrypoint="molmcp.skills.daily:Daily") + + def test_rejects_agent_with_entrypoint(self): + with pytest.raises(CatalogError): + _agent_spec(entrypoint="molmcp.agents.reviewer:Reviewer") + + def test_rejects_rule_with_entrypoint(self): + with pytest.raises(CatalogError): + _rule_spec(entrypoint="molmcp.rules.safety:Safety") + + def test_is_frozen(self): + spec = _skill_spec() + with pytest.raises(dataclasses.FrozenInstanceError): + spec.name = "other" # type: ignore[misc] + + def test_uses_slots(self): + spec = _skill_spec() + assert not hasattr(spec, "__dict__") + assert hasattr(ComponentSpec, "__slots__") + + +class TestBundleSpec: + def test_allowed_requires_is_the_two_capability_tokens(self): + assert ALLOWED_REQUIRES == frozenset({"provider-sdk", "harness-catalog"}) + + def test_constructs_daily_with_empty_requires(self): + spec = BundleSpec( + name="daily", + members=("skill.daily", "rule.safety"), + ) + assert spec.name == "daily" + assert spec.members == ("skill.daily", "rule.safety") + assert spec.requires == () + + def test_constructs_with_provider_sdk_requires(self): + spec = BundleSpec( + name="daily", + members=("skill.daily", "rule.safety"), + requires=("provider-sdk",), + ) + assert spec.requires == ("provider-sdk",) + + def test_rejects_empty_members(self): + with pytest.raises(CatalogError): + _bundle_spec(members=()) + + def test_rejects_member_without_kind_prefix(self): + with pytest.raises(CatalogError): + _bundle_spec(members=("daily",)) + + def test_rejects_bundle_member(self): + with pytest.raises(CatalogError): + _bundle_spec(members=("bundle.daily",)) + + def test_rejects_unknown_requires_token(self): + with pytest.raises(CatalogError): + _bundle_spec(requires=("not-a-capability",)) + + def test_rejects_name_with_underscore(self): + with pytest.raises(CatalogError): + _bundle_spec(name="daily_bundle") + + def test_is_frozen(self): + spec = _bundle_spec() + with pytest.raises(dataclasses.FrozenInstanceError): + spec.name = "other" # type: ignore[misc] + + def test_uses_slots(self): + spec = _bundle_spec() + assert not hasattr(spec, "__dict__") + assert hasattr(BundleSpec, "__slots__") diff --git a/tests/test_components/test_store.py b/tests/test_components/test_store.py new file mode 100644 index 0000000..941a01f --- /dev/null +++ b/tests/test_components/test_store.py @@ -0,0 +1,265 @@ +"""ImmutableGitStore — fake GitTransport, no DiscoveryEngine.""" + +from __future__ import annotations + +import inspect +import io +import json +import tarfile +from pathlib import Path + +import pytest + +from molmcp.components.git import extract_git_archive +from molmcp.components.store import ( + ImmutableGitStore, + ShaConflictError, + StoreError, + UnknownShaError, +) + +SHA_A = "a" * 40 +_OWNER = "acme" +_REPO = "widgets" +_HARNESS_TOML = "# harness\n" +_STORE_PY = ( + Path(__file__).resolve().parents[2] / "src" / "molmcp" / "components" / "store.py" +) + + +def _github_tarball(repo: str, sha: str) -> bytes: + """GitHub-style tar.gz whose inner directory is ``{repo}-{sha}/``.""" + prefix = f"{repo}-{sha}" + members = { + f"{prefix}/harness.toml": _HARNESS_TOML, + f"{prefix}/dummy.txt": "dummy\n", + } + buf = io.BytesIO() + with tarfile.open(fileobj=buf, mode="w:gz") as tar: + for name, content in members.items(): + data = content.encode("utf-8") + info = tarfile.TarInfo(name=name) + info.size = len(data) + tar.addfile(info, io.BytesIO(data)) + return buf.getvalue() + + +class _FakeGitTransport: + """``fetch_archive`` only; ``resolve_commit`` raises if the store calls it.""" + + def __init__(self, archive: bytes) -> None: + self._archive = archive + self.fetch_calls: list[tuple[str, str, str]] = [] + self.resolve_calls: list[tuple[str, str, str | None]] = [] + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + self.fetch_calls.append((owner, repo, sha)) + return self._archive + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + self.resolve_calls.append((owner, repo, ref)) + raise AssertionError("ImmutableGitStore must not call resolve_commit") + + +def _new_store(tmp_path: Path) -> tuple[ImmutableGitStore, _FakeGitTransport]: + transport = _FakeGitTransport(_github_tarball(_REPO, SHA_A)) + return ImmutableGitStore(tmp_path, transport), transport + + +def _published(tmp_path: Path) -> tuple[ImmutableGitStore, _FakeGitTransport]: + store, transport = _new_store(tmp_path) + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + return store, transport + + +def _plant_metadata_only(root: Path, sha: str) -> Path: + sha_dir = root / "commits" / sha + sha_dir.mkdir(parents=True) + path = sha_dir / "metadata.json" + path.write_text( + json.dumps({"sha": sha, "owner": _OWNER, "repo": _REPO}), + encoding="utf-8", + ) + return path + + +def _plant_tree_only(root: Path, sha: str) -> Path: + tree = root / "commits" / sha / "tree" + tree.mkdir(parents=True) + (tree / "harness.toml").write_text(_HARNESS_TOML, encoding="utf-8") + return tree + + +def _store_source() -> str: + return _STORE_PY.read_text(encoding="utf-8") + + +class TestImmutableGitStore: + def test_constructs_without_arguments_raises_type_error(self): + with pytest.raises(TypeError): + ImmutableGitStore() # type: ignore[call-arg] + + def test_constructs_without_root_raises_type_error(self): + transport = _FakeGitTransport(_github_tarball(_REPO, SHA_A)) + with pytest.raises(TypeError): + ImmutableGitStore(transport=transport) # type: ignore[call-arg] + + def test_constructs_without_transport_raises_type_error(self, tmp_path): + with pytest.raises(TypeError): + ImmutableGitStore(tmp_path) # type: ignore[call-arg] + + def test_constructs_with_root_none_raises_type_error(self): + transport = _FakeGitTransport(_github_tarball(_REPO, SHA_A)) + with pytest.raises(TypeError): + ImmutableGitStore(None, transport) # type: ignore[arg-type] + + def test_constructs_with_transport_none_raises_type_error(self, tmp_path): + with pytest.raises(TypeError): + ImmutableGitStore(tmp_path, None) # type: ignore[arg-type] + + def test_constructs_with_root_and_transport_positionally(self, tmp_path): + params = inspect.signature(ImmutableGitStore).parameters + assert list(params) == ["root", "transport"] + assert params["root"].default is inspect.Parameter.empty + assert params["transport"].default is inspect.Parameter.empty + transport = _FakeGitTransport(_github_tarball(_REPO, SHA_A)) + store = ImmutableGitStore(tmp_path, transport) + assert isinstance(store, ImmutableGitStore) + + def test_publish_writes_owner_and_repo_from_kwargs(self, tmp_path): + store, _transport = _new_store(tmp_path) + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + payload = json.loads( + (tmp_path / "commits" / SHA_A / "metadata.json").read_text(encoding="utf-8") + ) + assert payload["owner"] == _OWNER + assert payload["repo"] == _REPO + assert payload["owner"] != _REPO + + def test_publish_places_harness_toml_at_flattened_tree_path(self, tmp_path): + store, _transport = _new_store(tmp_path) + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + tree = store.tree_path(SHA_A) + assert tree == tmp_path / "commits" / SHA_A / "tree" + harness = tree / "harness.toml" + assert harness.is_file() + assert harness.read_text(encoding="utf-8") == _HARNESS_TOML + assert not (tree / f"{_REPO}-{SHA_A}").exists() + + def test_has_is_true_after_complete_publish(self, tmp_path): + store, _transport = _new_store(tmp_path) + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + assert store.has(SHA_A) is True + + def test_publish_returns_path_where_tree_path_works(self, tmp_path): + store, _transport = _new_store(tmp_path) + returned = store.publish(SHA_A, owner=_OWNER, repo=_REPO) + tree = store.tree_path(SHA_A) + sha_dir = tmp_path / "commits" / SHA_A + assert isinstance(returned, Path) + assert returned in {tree, sha_dir} + assert tree == sha_dir / "tree" + assert tree.is_dir() + + def test_republish_same_provenance_does_not_fetch_archive(self, tmp_path): + store, transport = _published(tmp_path) + assert len(transport.fetch_calls) == 1 + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + assert len(transport.fetch_calls) == 1 + + def test_republish_same_provenance_does_not_replace_tree(self, tmp_path): + store, _transport = _published(tmp_path) + sentinel = store.tree_path(SHA_A) / "sentinel.txt" + sentinel.write_text("planted", encoding="utf-8") + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + assert sentinel.is_file() + assert sentinel.read_text(encoding="utf-8") == "planted" + + def test_publish_same_sha_different_owner_raises_sha_conflict_error(self, tmp_path): + store, _transport = _published(tmp_path) + with pytest.raises(ShaConflictError): + store.publish(SHA_A, owner="other", repo=_REPO) + + def test_publish_same_sha_different_repo_raises_sha_conflict_error(self, tmp_path): + store, _transport = _published(tmp_path) + with pytest.raises(ShaConflictError): + store.publish(SHA_A, owner=_OWNER, repo="gadgets") + + def test_publish_conflict_leaves_tree_unchanged(self, tmp_path): + store, _transport = _published(tmp_path) + sentinel = store.tree_path(SHA_A) / "sentinel.txt" + sentinel.write_text("planted", encoding="utf-8") + with pytest.raises(ShaConflictError): + store.publish(SHA_A, owner="other", repo=_REPO) + assert sentinel.read_text(encoding="utf-8") == "planted" + + def test_has_is_false_when_only_metadata_exists(self, tmp_path): + _plant_metadata_only(tmp_path, SHA_A) + store, _transport = _new_store(tmp_path) + assert store.has(SHA_A) is False + + def test_tree_path_raises_unknown_sha_when_only_metadata_exists(self, tmp_path): + _plant_metadata_only(tmp_path, SHA_A) + store, _transport = _new_store(tmp_path) + with pytest.raises(UnknownShaError): + store.tree_path(SHA_A) + + def test_has_is_false_when_only_tree_exists(self, tmp_path): + _plant_tree_only(tmp_path, SHA_A) + store, _transport = _new_store(tmp_path) + assert store.has(SHA_A) is False + + def test_tree_path_raises_unknown_sha_when_only_tree_exists(self, tmp_path): + _plant_tree_only(tmp_path, SHA_A) + store, _transport = _new_store(tmp_path) + with pytest.raises(UnknownShaError): + store.tree_path(SHA_A) + + def test_publish_completes_metadata_only_directory(self, tmp_path): + _plant_metadata_only(tmp_path, SHA_A) + store, transport = _new_store(tmp_path) + store.publish(SHA_A, owner=_OWNER, repo=_REPO) + assert len(transport.fetch_calls) == 1 + assert store.has(SHA_A) is True + harness = store.tree_path(SHA_A) / "harness.toml" + assert harness.is_file() + assert harness.read_text(encoding="utf-8") == _HARNESS_TOML + + def test_has_is_false_for_never_published_sha(self, tmp_path): + store, _transport = _new_store(tmp_path) + assert store.has(SHA_A) is False + + def test_tree_path_raises_unknown_sha_for_never_published_sha(self, tmp_path): + store, _transport = _new_store(tmp_path) + with pytest.raises(UnknownShaError): + store.tree_path(SHA_A) + + def test_store_source_does_not_contain_materialize(self): + assert "materialize" not in _store_source() + + def test_store_source_does_not_contain_resolve_commit(self): + assert "resolve_commit" not in _store_source() + + def test_store_source_uses_extract_git_archive(self): + assert extract_git_archive.__name__ in _store_source() + + def test_publish_does_not_create_refs_directory(self, tmp_path): + _published(tmp_path) + assert not (tmp_path / "refs").exists() + + def test_publish_does_not_create_pointers_directory(self, tmp_path): + _published(tmp_path) + assert not (tmp_path / "pointers").exists() + + def test_publish_does_not_write_pointer_files(self, tmp_path): + _published(tmp_path) + assert {path.name for path in tmp_path.iterdir()} == {"commits"} + + def test_layer_errors_subclass_store_error(self): + assert issubclass(StoreError, Exception) + assert issubclass(UnknownShaError, StoreError) + assert issubclass(ShaConflictError, StoreError) + + def test_publish_does_not_call_resolve_commit(self, tmp_path): + _store, transport = _published(tmp_path) + assert transport.resolve_calls == [] diff --git a/tests/test_config_scope.py b/tests/test_config_scope.py index 88a342b..b4e4de0 100644 --- a/tests/test_config_scope.py +++ b/tests/test_config_scope.py @@ -11,20 +11,10 @@ import json -import pytest - from molmcp import settings as st from molmcp.config import AppConfig, load_config -@pytest.fixture -def home(tmp_path, monkeypatch): - fake = tmp_path / "home" - fake.mkdir() - monkeypatch.setattr(st.Path, "home", staticmethod(lambda: fake)) - return fake - - def _write_settings(path, data) -> None: path.parent.mkdir(parents=True, exist_ok=True) path.write_text(json.dumps(data), encoding="utf-8") diff --git a/tests/test_evolution/__init__.py b/tests/test_evolution/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_evolution/test_evaluate.py b/tests/test_evolution/test_evaluate.py new file mode 100644 index 0000000..e6ef14f --- /dev/null +++ b/tests/test_evolution/test_evaluate.py @@ -0,0 +1,956 @@ +"""Held-out challenger gate — four independent metrics, never a total score. + +Mirrors ``src/molmcp/evolution/evaluate.py``; one class per public behaviour +(``Metrics`` and ``EvaluationReport`` the value objects, ``evaluate`` the +function). ``EvalCase``, ``ContractOutcome`` and the ``Challenger`` / +``ContractRunner`` / ``ReplayFn`` protocols are exercised *through* those +three: they are literals and seams a caller builds, and a test that only +constructed them would pin no behaviour. + +Note the naming this module inherits. The duck-typed protocol for "the +checkout under evaluation" is ``Challenger``, because ``Candidate`` is +already a dataclass in :mod:`molmcp.evolution` (spec 10: a proposed patch). +The *report field* is still ``candidate_sha`` — only the protocol was +renamed. + +Four disciplines are pinned here that no single assertion makes obvious. + +*The float mean decides, the rounded mean is only stored.* Three of the four +metrics are ints, and the report keeps ``round(mean)``; the worse/better +comparison happens on the un-rounded mean. The rounding-trap tests build +sides whose float means differ while their rounded ints are equal, and each +one pairs that hidden move with a visible move in another field, so an +implementation that compares the rounded ints returns a *different verdict* +rather than the same one by luck. + +*The regression contract short-circuits.* When ``runner`` says no, ``replay`` +is never called and both sides' metrics are zero. The fixture for that test +hands the fake replay a strictly better challenger, so an implementation that +runs the replay anyway accepts instead of rejecting. + +*Worse beats better, and the first worse field names the reason.* The four +fields are compared in order ``tool_errors`` → ``call_count`` → ``tokens`` → +``latency_s``; a gain in one field never offsets a regression in another, +because there is no total to trade them in. + +*The seams have no default.* ``runner`` and ``replay`` are keyword-only with +no default at all — a default would have to be a real host, which would drag +MCP into this leaf. The fakes here decide which side they were asked about +from the *type* of ``target``: a ``str`` is the champion sha, a ``Path`` is +the challenger tree, so passing the wrong one returns the wrong numbers. + +Nothing here reads a clock, the environment, the network, or the filesystem. +``_TREE`` is a literal path that is never created: ``evaluate`` hands it to +the seams, it does not stat it. The only file read is ``evaluate.py`` itself, +and only to prove what it does not import. +""" + +from __future__ import annotations + +import ast +import dataclasses +import inspect +from collections.abc import Mapping, Sequence +from pathlib import Path + +import pytest + +from molmcp.evolution.evaluate import ( + ACCEPTED, + DEFAULT_SEEDS, + DROP_CALL_COUNT, + DROP_LATENCY_S, + DROP_TOKENS, + DROP_TOOL_ERRORS, + NO_PRACTICAL_GAIN, + REGRESSION_FAILED, + WORSE_CALL_COUNT, + WORSE_LATENCY, + WORSE_TOKENS, + WORSE_TOOL_ERRORS, + Challenger, + ContractOutcome, + ContractRunner, + EvalCase, + EvaluationError, + EvaluationReport, + Metrics, + ReplayFn, + evaluate, +) + +_REPO = Path(__file__).resolve().parents[2] +_EVALUATE = _REPO / "src" / "molmcp" / "evolution" / "evaluate.py" + +#: The dotted package the module under test lives in, used to resolve the +#: relative imports its purity check has to see through. +_PACKAGE_PARTS: tuple[str, ...] = ("molmcp", "evolution") + +#: The two shas the report echoes back, and the challenger's own identity. +#: Full shas: the spec says ``champion_sha`` is the complete string. +_CHAMPION_SHA = "a" * 40 +_CHALLENGER_SHA = "b" * 40 +_COMPONENT = "daily-pack-skill" + +#: Never created on disk. ``evaluate`` passes it to ``runner`` and ``replay`` +#: and must not touch it, so a test run needs no ``tmp_path`` at all. +_TREE = Path("/challenger/tree") + +#: Held-out cases go to ``replay``; regression cases go to ``runner``. Two +#: distinct sequences, so a swap shows up as the wrong ids on the wrong seam. +_HELD_OUT: tuple[EvalCase, ...] = (EvalCase(id="held-1"), EvalCase(id="held-2")) +_REGRESSION: tuple[EvalCase, ...] = (EvalCase(id="reg-1"),) +_HELD_OUT_IDS: tuple[str, ...] = ("held-1", "held-2") +_REGRESSION_IDS: tuple[str, ...] = ("reg-1",) + +#: ``Metrics`` fields, in the order the spec's value-object table lists them — +#: which is also the order the worse-field scan must use. +_METRICS_FIELDS: tuple[str, ...] = ( + "tool_errors", + "call_count", + "tokens", + "latency_s", +) + +#: ``EvaluationReport`` fields, in the spec's order. +_REPORT_FIELDS: tuple[str, ...] = ( + "accepted", + "reason", + "candidate_sha", + "champion_sha", + "seeds", + "regression_passed", + "champion_metrics", + "challenger_metrics", +) + +#: Names that would turn four independent metrics back into one number. +#: Forbidden as attributes, not merely unused. +_ABSENT_ON_METRICS: tuple[str, ...] = ( + "score", + "total", + "weighted_sum", + "composite", + "f1", + "rank", +) + +#: Names that would make the report a pointer writer. Promotion is spec 12; +#: this report is a verdict and nothing else. +_ABSENT_ON_REPORT: tuple[str, ...] = ( + "pointer", + "active", + "previous", + "stage", + "score", +) + +#: The seven frozen reason literals, spelling included. 12-promote and +#: 13-ci-gate match on these strings; a synonym is a break. +_REASONS: tuple[tuple[str, str], ...] = ( + (ACCEPTED, "accepted"), + (REGRESSION_FAILED, "regression_failed"), + (WORSE_TOOL_ERRORS, "worse_tool_errors"), + (WORSE_CALL_COUNT, "worse_call_count"), + (WORSE_TOKENS, "worse_tokens"), + (WORSE_LATENCY, "worse_latency"), + (NO_PRACTICAL_GAIN, "no_practical_gain"), +) + +#: Layers this leaf may not reach for. The first four are the runtime it must +#: stay out of; the last two are the MCP machinery a default seam would drag +#: in. +_FORBIDDEN_IMPORT_PREFIXES: tuple[str, ...] = ( + "molmcp.cli", + "molmcp.server", + "molmcp.providers", + "molmcp.collection", + "fastmcp", + "mcp", +) + + +def _metrics( + *, + tool_errors: int = 2, + call_count: int = 10, + tokens: int = 100, + latency_s: float = 1.0, +) -> Metrics: + """The baseline reading, with at most one field swapped out.""" + return Metrics( + tool_errors=tool_errors, + call_count=call_count, + tokens=tokens, + latency_s=latency_s, + ) + + +#: What both sides read when a test is not saying anything about a field. +_BASELINE = _metrics() + +#: Strictly better than ``_BASELINE`` on one field, equal on the rest. +_BETTER = _metrics(tool_errors=1) + +#: What the report must carry when the regression contract short-circuits. +_ZERO = _metrics(tool_errors=0, call_count=0, tokens=0, latency_s=0.0) + +#: Arithmetic-mean comparison tolerance. ``latency_s`` is a mean of literal +#: floats, not a measured quantity, so only representation error is allowed. +_LATENCY_TOL = 1e-12 + + +class FakeChallenger: + """Stand-in for the ``Challenger`` protocol — three read-only attributes. + + Deliberately not a subclass: the protocol is duck-typed, and a fake that + inherited it would hide a rename of any of the three names. + """ + + def __init__( + self, + sha: str = _CHALLENGER_SHA, + component: str = _COMPONENT, + affected_paths: Sequence[str] = ("skills/daily/pack.md",), + ) -> None: + self.sha: str = sha + self.component: str = component + self.affected_paths: tuple[str, ...] = tuple(affected_paths) + + +class FakeRunner: + """Stand-in for ``ContractRunner`` — records ``run``, replays one outcome.""" + + def __init__(self, outcome: ContractOutcome) -> None: + self._outcome = outcome + self.calls: list[tuple[Path, tuple[str, ...]]] = [] + + def run(self, tree: Path, cases: Sequence[EvalCase]) -> ContractOutcome: + self.calls.append((tree, tuple(case.id for case in cases))) + return self._outcome + + +class FakeReplay: + """Stand-in for ``ReplayFn`` — one metrics table per side, keyed by seed. + + ``target`` decides the side: a ``str`` is the champion sha, a ``Path`` is + the challenger tree. An implementation that hands over the wrong type + reads the wrong side's numbers, so the seam's types are pinned by every + verdict here as well as by ``test_replay_gets_a_sha_then_a_tree``. + """ + + def __init__( + self, + champion: Mapping[int, Metrics], + challenger: Mapping[int, Metrics], + ) -> None: + self._champion = dict(champion) + self._challenger = dict(challenger) + self.calls: list[tuple[str | Path, tuple[str, ...], int]] = [] + + def __call__( + self, target: str | Path, cases: Sequence[EvalCase], seed: int + ) -> Metrics: + self.calls.append((target, tuple(case.id for case in cases), seed)) + table = self._challenger if isinstance(target, Path) else self._champion + if seed not in table: + raise AssertionError( + f"replay called with unexpected seed {seed!r}; " + f"table has {sorted(table)}" + ) + return table[seed] + + @property + def seeds_seen(self) -> list[int]: + return [seed for _, _, seed in self.calls] + + +def _challenger(sha: str = _CHALLENGER_SHA) -> Challenger: + return FakeChallenger(sha=sha) + + +def _runner(passed: bool = True, failed_case_ids: Sequence[str] = ()) -> FakeRunner: + return FakeRunner( + ContractOutcome(passed=passed, failed_case_ids=tuple(failed_case_ids)) + ) + + +def _flat(metrics: Metrics, seeds: Sequence[int] = DEFAULT_SEEDS) -> dict[int, Metrics]: + """One reading repeated for every seed.""" + return {seed: metrics for seed in seeds} + + +def _per_seed( + readings: Sequence[Metrics], seeds: Sequence[int] = DEFAULT_SEEDS +) -> dict[int, Metrics]: + """One reading per seed, paired positionally.""" + return dict(zip(seeds, readings, strict=True)) + + +def _replay( + champion: Mapping[int, Metrics] | None = None, + challenger: Mapping[int, Metrics] | None = None, +) -> FakeReplay: + """Both sides flat on ``_BASELINE`` unless a test says otherwise.""" + return FakeReplay( + _flat(_BASELINE) if champion is None else champion, + _flat(_BASELINE) if challenger is None else challenger, + ) + + +def _evaluate( + runner: FakeRunner, + replay: FakeReplay, + *, + seeds: Sequence[int] | None = None, + held_out_cases: Sequence[EvalCase] = _HELD_OUT, + regression_cases: Sequence[EvalCase] = _REGRESSION, + challenger_tree: Path = _TREE, +) -> EvaluationReport: + """Call ``evaluate`` by keyword, omitting ``seeds`` entirely when ``None``.""" + if seeds is None: + return evaluate( + challenger=_challenger(), + challenger_tree=challenger_tree, + champion_sha=_CHAMPION_SHA, + held_out_cases=held_out_cases, + regression_cases=regression_cases, + runner=runner, + replay=replay, + ) + return evaluate( + challenger=_challenger(), + challenger_tree=challenger_tree, + champion_sha=_CHAMPION_SHA, + held_out_cases=held_out_cases, + regression_cases=regression_cases, + runner=runner, + replay=replay, + seeds=seeds, + ) + + +def _kwargs_without( + seam: str, runner: FakeRunner, replay: FakeReplay +) -> dict[str, object]: + """Every argument ``evaluate`` needs, minus one injected seam.""" + kwargs: dict[str, object] = { + "challenger": _challenger(), + "challenger_tree": _TREE, + "champion_sha": _CHAMPION_SHA, + "held_out_cases": _HELD_OUT, + "regression_cases": _REGRESSION, + "runner": runner, + "replay": replay, + } + del kwargs[seam] + return kwargs + + +def _valid_report( + *, + accepted: bool = True, + reason: str = ACCEPTED, + candidate_sha: str = _CHALLENGER_SHA, + champion_sha: str = _CHAMPION_SHA, + seeds: tuple[int, ...] = DEFAULT_SEEDS, + regression_passed: bool = True, + champion_metrics: Metrics = _BASELINE, + challenger_metrics: Metrics = _BETTER, +) -> EvaluationReport: + """An accepted report, with at most one invariant swapped out.""" + return EvaluationReport( + accepted=accepted, + reason=reason, + candidate_sha=candidate_sha, + champion_sha=champion_sha, + seeds=seeds, + regression_passed=regression_passed, + champion_metrics=champion_metrics, + challenger_metrics=challenger_metrics, + ) + + +def _evaluate_source() -> str: + assert _EVALUATE.is_file(), f"{_EVALUATE} does not exist yet" + return _EVALUATE.read_text(encoding="utf-8") + + +def _resolved_module(node: ast.ImportFrom) -> str: + """The dotted module *node* names, with a relative import made absolute.""" + if not node.level: + return node.module or "" + kept = len(_PACKAGE_PARTS) - node.level + 1 + base = ".".join(_PACKAGE_PARTS[:kept]) if kept > 0 else "" + if not node.module: + return base + return f"{base}.{node.module}" if base else node.module + + +def _module_level_imports(tree: ast.Module) -> set[str]: + """Modules imported at module level — not inside a function or a block.""" + modules: set[str] = set() + for node in tree.body: + if isinstance(node, ast.Import): + modules.update(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + module = _resolved_module(node) + modules.add(module) + modules.update(f"{module}.{alias.name}" for alias in node.names) + return modules + + +#: One field worse than ``_BASELINE``, the rest equal → the reason it names. +_WORSE_CASES = ( + pytest.param(_metrics(tool_errors=3), WORSE_TOOL_ERRORS, id="tool_errors"), + pytest.param(_metrics(call_count=11), WORSE_CALL_COUNT, id="call_count"), + pytest.param(_metrics(tokens=101), WORSE_TOKENS, id="tokens"), + pytest.param(_metrics(latency_s=1.5), WORSE_LATENCY, id="latency_s"), +) + +#: Several fields worse at once → the *first* in field order names the reason. +#: An implementation scanning in any other order fails at least one of these. +_PRECEDENCE_CASES = ( + pytest.param( + _metrics(tool_errors=3, call_count=11), + WORSE_TOOL_ERRORS, + id="tool_errors-before-call_count", + ), + pytest.param( + _metrics(tool_errors=3, latency_s=1.5), + WORSE_TOOL_ERRORS, + id="tool_errors-before-latency_s", + ), + pytest.param( + _metrics(call_count=11, tokens=101), + WORSE_CALL_COUNT, + id="call_count-before-tokens", + ), + pytest.param( + _metrics(tokens=101, latency_s=1.5), + WORSE_TOKENS, + id="tokens-before-latency_s", + ), + pytest.param( + _metrics(tool_errors=3, call_count=11, tokens=101, latency_s=1.5), + WORSE_TOOL_ERRORS, + id="all-four-worse", + ), +) + +#: One field better than ``_BASELINE``, the rest equal → accepted. +_BETTER_CASES = ( + pytest.param(_metrics(tool_errors=1), id="tool_errors"), + pytest.param(_metrics(call_count=9), id="call_count"), + pytest.param(_metrics(tokens=99), id="tokens"), + pytest.param(_metrics(latency_s=0.5), id="latency_s"), +) + +#: Report shapes the ``__post_init__`` must make unrepresentable. +_ILLEGAL_REPORTS = ( + pytest.param( + {"accepted": True, "regression_passed": False}, + id="accepted-while-regression-failed", + ), + pytest.param( + {"accepted": True, "reason": REGRESSION_FAILED}, + id="accepted-reading-regression_failed", + ), + pytest.param( + {"accepted": True, "reason": WORSE_TOOL_ERRORS}, + id="accepted-reading-worse_tool_errors", + ), + pytest.param( + {"accepted": True, "reason": NO_PRACTICAL_GAIN}, + id="accepted-reading-no_practical_gain", + ), + pytest.param( + {"accepted": False, "reason": ACCEPTED}, + id="rejected-reading-accepted", + ), +) + +#: Report shapes that must stay representable — in particular a rejection +#: whose regression suite *passed*, which is every step-2 and step-3 verdict. +_LEGAL_REPORTS = ( + pytest.param( + {"accepted": True, "reason": ACCEPTED, "regression_passed": True}, + id="accepted", + ), + pytest.param( + {"accepted": False, "reason": REGRESSION_FAILED, "regression_passed": False}, + id="regression-failed", + ), + pytest.param( + {"accepted": False, "reason": WORSE_TOKENS, "regression_passed": True}, + id="worse-though-regression-passed", + ), + pytest.param( + {"accepted": False, "reason": NO_PRACTICAL_GAIN, "regression_passed": True}, + id="no-practical-gain", + ), +) + + +class TestMetrics: + def test_carries_the_four_readings_it_was_given(self) -> None: + metrics = _metrics(tool_errors=3, call_count=12, tokens=345, latency_s=2.5) + + assert metrics.tool_errors == 3 + assert metrics.call_count == 12 + assert metrics.tokens == 345 + assert metrics.latency_s == pytest.approx(2.5, abs=_LATENCY_TOL) + + def test_field_names_are_the_value_object_table_in_order(self) -> None: + names = tuple(field.name for field in dataclasses.fields(Metrics)) + + assert names == _METRICS_FIELDS + + def test_has_exactly_four_fields(self) -> None: + assert len(dataclasses.fields(Metrics)) == 4 + + @pytest.mark.parametrize("name", _ABSENT_ON_METRICS) + def test_carries_no_composite_number(self, name: str) -> None: + """Four independent metrics, never summed, weighted, or ranked.""" + assert not hasattr(Metrics, name) + assert not hasattr(_metrics(), name) + + def test_the_class_carries_no_helper_beyond_its_four_fields(self) -> None: + """A weighted-sum method would show up here as a fifth public name.""" + public = {name for name in vars(Metrics) if not name.startswith("_")} + + assert public == set(_METRICS_FIELDS) + + @pytest.mark.parametrize("field_name", _METRICS_FIELDS) + def test_is_frozen(self, field_name: str) -> None: + metrics = _metrics() + + with pytest.raises(dataclasses.FrozenInstanceError): + setattr(metrics, field_name, 99) + + def test_uses_slots(self) -> None: + assert hasattr(Metrics, "__slots__") + assert not hasattr(_metrics(), "__dict__") + + def test_equal_readings_compare_equal(self) -> None: + """Verdicts are read off equality; a reading is its four numbers.""" + assert _metrics() == _metrics() + assert _metrics() != _metrics(tokens=101) + + +class TestEvaluationReport: + def test_carries_the_eight_fields_it_was_given(self) -> None: + report = _valid_report() + + assert report.accepted is True + assert report.reason == ACCEPTED + assert report.candidate_sha == _CHALLENGER_SHA + assert report.champion_sha == _CHAMPION_SHA + assert report.seeds == DEFAULT_SEEDS + assert report.regression_passed is True + assert report.champion_metrics == _BASELINE + assert report.challenger_metrics == _BETTER + + def test_field_names_are_the_value_object_table_in_order(self) -> None: + """The protocol was renamed to ``Challenger``; the field was not.""" + names = tuple(field.name for field in dataclasses.fields(EvaluationReport)) + + assert names == _REPORT_FIELDS + + def test_has_exactly_eight_fields(self) -> None: + assert len(dataclasses.fields(EvaluationReport)) == 8 + + @pytest.mark.parametrize("name", _ABSENT_ON_REPORT) + def test_carries_no_pointer_or_score(self, name: str) -> None: + """A verdict, not a promotion: moving the pointer is spec 12's job.""" + assert not hasattr(EvaluationReport, name) + assert not hasattr(_valid_report(), name) + assert name not in _REPORT_FIELDS + + @pytest.mark.parametrize("overrides", _LEGAL_REPORTS) + def test_consistent_verdicts_are_representable( + self, overrides: dict[str, object] + ) -> None: + report = _valid_report(**overrides) + + assert report.accepted is overrides["accepted"] + assert report.reason == overrides["reason"] + assert report.regression_passed is overrides["regression_passed"] + + @pytest.mark.parametrize("overrides", _ILLEGAL_REPORTS) + def test_inconsistent_verdicts_are_unrepresentable( + self, overrides: dict[str, object] + ) -> None: + with pytest.raises(EvaluationError): + _valid_report(**overrides) + + @pytest.mark.parametrize("field_name", _REPORT_FIELDS) + def test_is_frozen(self, field_name: str) -> None: + report = _valid_report() + + with pytest.raises(dataclasses.FrozenInstanceError): + setattr(report, field_name, "mutated") + + def test_uses_slots(self) -> None: + assert hasattr(EvaluationReport, "__slots__") + assert not hasattr(_valid_report(), "__dict__") + + +class TestEvaluate: + # -- constants --------------------------------------------------------- + + def test_the_default_seeds_are_frozen_at_one_two_three(self) -> None: + assert DEFAULT_SEEDS == (1, 2, 3) + assert isinstance(DEFAULT_SEEDS, tuple) + + def test_the_drop_thresholds_are_zero(self) -> None: + """No noise band: any move in the wrong direction is a regression.""" + assert DROP_TOOL_ERRORS == 0 + assert DROP_CALL_COUNT == 0 + assert DROP_TOKENS == 0 + assert DROP_LATENCY_S == 0.0 + assert isinstance(DROP_LATENCY_S, float) + + @pytest.mark.parametrize( + ("literal", "expected"), _REASONS, ids=[text for _, text in _REASONS] + ) + def test_the_reason_literals_are_frozen(self, literal: str, expected: str) -> None: + assert literal == expected + + def test_the_reasons_are_seven_distinct_strings(self) -> None: + assert len({literal for literal, _ in _REASONS}) == 7 + + def test_the_error_is_a_value_error(self) -> None: + assert issubclass(EvaluationError, ValueError) + + # -- the injected seams ------------------------------------------------ + + def test_the_fakes_satisfy_the_injected_seams(self) -> None: + """The seam shapes this module injects, stated as types once.""" + runner: ContractRunner = _runner() + replay: ReplayFn = _replay() + + assert runner.run(_TREE, _REGRESSION).passed is True + assert replay(_CHAMPION_SHA, _HELD_OUT, 1) == _BASELINE + + def test_runner_and_replay_are_keyword_only_without_a_default(self) -> None: + """A default seam would have to be a real host — that is 13's job.""" + params = inspect.signature(evaluate).parameters + + for name in ("runner", "replay"): + assert params[name].kind is inspect.Parameter.KEYWORD_ONLY + assert params[name].default is inspect.Parameter.empty + assert params["seeds"].kind is inspect.Parameter.KEYWORD_ONLY + + @pytest.mark.parametrize("seam", ["runner", "replay"]) + def test_omitting_a_seam_is_a_type_error(self, seam: str) -> None: + runner = _runner() + replay = _replay() + + with pytest.raises(TypeError): + evaluate(**_kwargs_without(seam, runner, replay)) + + assert runner.calls == [] + assert replay.calls == [] + + # -- refusals ---------------------------------------------------------- + + def test_empty_seeds_raises_before_anything_runs(self) -> None: + """Immediately means immediately: no contract run, and no report.""" + runner = _runner() + replay = _replay() + + with pytest.raises(EvaluationError): + _evaluate(runner, replay, seeds=()) + + assert runner.calls == [] + assert replay.calls == [] + + def test_empty_held_out_cases_raises_before_anything_runs(self) -> None: + """This is the held-out gate, not a contract-only channel.""" + runner = _runner() + replay = _replay() + + with pytest.raises(EvaluationError): + _evaluate(runner, replay, held_out_cases=()) + + assert runner.calls == [] + assert replay.calls == [] + + def test_no_regression_cases_yet_is_legal_and_passes(self) -> None: + replay = _replay(challenger=_flat(_BETTER)) + + report = _evaluate(_runner(), replay, regression_cases=()) + + assert report.regression_passed is True + assert report.accepted is True + assert report.reason == ACCEPTED + + # -- step 1: the regression contract short-circuits --------------------- + + def test_a_failed_contract_rejects_without_replaying(self) -> None: + """The challenger table here is strictly better, and never read.""" + runner = _runner(passed=False, failed_case_ids=("reg-1",)) + replay = _replay(challenger=_flat(_BETTER)) + + report = _evaluate(runner, replay) + + assert report.accepted is False + assert report.reason == REGRESSION_FAILED + assert report.regression_passed is False + assert report.champion_metrics == _ZERO + assert report.challenger_metrics == _ZERO + assert replay.calls == [] + assert report.candidate_sha == _CHALLENGER_SHA + assert report.champion_sha == _CHAMPION_SHA + + def test_the_contract_runs_on_the_tree_with_the_regression_cases(self) -> None: + runner = _runner() + + _evaluate(runner, _replay(challenger=_flat(_BETTER))) + + assert runner.calls == [(_TREE, _REGRESSION_IDS)] + + # -- step 2: the replay ------------------------------------------------ + + def test_replay_runs_once_per_seed_for_each_side(self) -> None: + replay = _replay(challenger=_flat(_BETTER)) + + _evaluate(_runner(), replay) + + assert len(replay.calls) == 2 * len(DEFAULT_SEEDS) + assert sorted(replay.seeds_seen) == sorted(DEFAULT_SEEDS + DEFAULT_SEEDS) + + def test_replay_gets_a_sha_then_a_tree(self) -> None: + """Champion by full sha, challenger by checked-out tree.""" + replay = _replay(challenger=_flat(_BETTER)) + + _evaluate(_runner(), replay) + + targets = [target for target, _, _ in replay.calls] + champion_targets = [t for t in targets if isinstance(t, str)] + challenger_targets = [t for t in targets if isinstance(t, Path)] + + assert champion_targets == [_CHAMPION_SHA] * len(DEFAULT_SEEDS) + assert challenger_targets == [_TREE] * len(DEFAULT_SEEDS) + assert len(targets) == len(champion_targets) + len(challenger_targets) + + def test_replay_gets_the_held_out_cases_not_the_regression_ones(self) -> None: + replay = _replay(challenger=_flat(_BETTER)) + + _evaluate(_runner(), replay) + + assert [ids for _, ids, _ in replay.calls] == [_HELD_OUT_IDS] * 6 + + @pytest.mark.parametrize(("challenger_metrics", "reason"), _WORSE_CASES) + def test_one_worse_field_rejects_and_names_itself( + self, challenger_metrics: Metrics, reason: str + ) -> None: + replay = _replay(challenger=_flat(challenger_metrics)) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == reason + + @pytest.mark.parametrize(("challenger_metrics", "reason"), _PRECEDENCE_CASES) + def test_the_first_worse_field_in_order_names_the_reason( + self, challenger_metrics: Metrics, reason: str + ) -> None: + replay = _replay(challenger=_flat(challenger_metrics)) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == reason + + def test_a_gain_never_offsets_a_regression(self) -> None: + """No total score means no trade: the worse field still decides.""" + replay = _replay(challenger=_flat(_metrics(tool_errors=1, latency_s=1.5))) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == WORSE_LATENCY + + # -- steps 3 and 4: the verdict ---------------------------------------- + + def test_no_move_at_all_is_not_a_gain(self) -> None: + replay = _replay() + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == NO_PRACTICAL_GAIN + assert report.regression_passed is True + + @pytest.mark.parametrize("challenger_metrics", _BETTER_CASES) + def test_one_better_field_and_no_worse_one_accepts( + self, challenger_metrics: Metrics + ) -> None: + replay = _replay(challenger=_flat(challenger_metrics)) + + report = _evaluate(_runner(), replay) + + assert report.accepted is True + assert report.reason == ACCEPTED + assert report.regression_passed is True + assert report.candidate_sha == _CHALLENGER_SHA + assert report.champion_sha == _CHAMPION_SHA + + # -- the report -------------------------------------------------------- + + def test_the_report_records_the_default_seeds_when_none_are_given(self) -> None: + report = _evaluate(_runner(), _replay(challenger=_flat(_BETTER))) + + assert report.seeds == DEFAULT_SEEDS + + def test_the_report_records_the_seeds_actually_used(self) -> None: + seeds = [7, 11] + replay = FakeReplay(_flat(_BASELINE, seeds), _flat(_BETTER, seeds)) + + report = _evaluate(_runner(), replay, seeds=seeds) + + assert report.seeds == (7, 11) + assert isinstance(report.seeds, tuple) + assert sorted(replay.seeds_seen) == [7, 7, 11, 11] + assert report.accepted is True + + def test_the_reports_metrics_are_the_seed_means(self) -> None: + """Int fields keep ``round(mean)``; ``latency_s`` keeps the mean.""" + replay = _replay( + champion=_per_seed( + ( + _metrics(tool_errors=2, call_count=10, tokens=100, latency_s=1.0), + _metrics(tool_errors=2, call_count=11, tokens=100, latency_s=1.0), + _metrics(tool_errors=3, call_count=11, tokens=101, latency_s=1.3), + ) + ), + challenger=_flat( + _metrics(tool_errors=0, call_count=5, tokens=50, latency_s=0.5) + ), + ) + + report = _evaluate(_runner(), replay) + + assert report.champion_metrics.tool_errors == 2 # mean 2.333… + assert report.champion_metrics.call_count == 11 # mean 10.667… + assert report.champion_metrics.tokens == 100 # mean 100.333… + assert report.champion_metrics.latency_s == pytest.approx(1.1, abs=_LATENCY_TOL) + assert report.challenger_metrics == _metrics( + tool_errors=0, call_count=5, tokens=50, latency_s=0.5 + ) + assert report.accepted is True + + # -- the rounding trap ------------------------------------------------- + + def test_a_regression_hidden_by_rounding_still_rejects(self) -> None: + """``call_count`` means 10.0 vs 10.333…; both round to 10. + + ``tokens`` improves, so an implementation that compares the *rounded* + ints sees one gain and no regression and accepts this fixture. + """ + replay = _replay( + champion=_flat(_metrics(call_count=10, tokens=100)), + challenger=_per_seed( + ( + _metrics(call_count=10, tokens=90), + _metrics(call_count=10, tokens=90), + _metrics(call_count=11, tokens=90), + ) + ), + ) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == WORSE_CALL_COUNT + assert report.champion_metrics.call_count == 10 + assert report.challenger_metrics.call_count == 10 + + def test_a_tool_error_regression_hidden_by_rounding_still_rejects(self) -> None: + """``tool_errors`` means 0.0 vs 0.333…; both round to 0.""" + replay = _replay( + champion=_flat(_metrics(tool_errors=0, tokens=100)), + challenger=_per_seed( + ( + _metrics(tool_errors=0, tokens=90), + _metrics(tool_errors=0, tokens=90), + _metrics(tool_errors=1, tokens=90), + ) + ), + ) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == WORSE_TOOL_ERRORS + assert report.champion_metrics.tool_errors == 0 + assert report.challenger_metrics.tool_errors == 0 + + def test_a_token_regression_hidden_by_rounding_still_rejects(self) -> None: + """``tokens`` means 100.0 vs 100.333…; both round to 100.""" + replay = _replay( + champion=_flat(_metrics(tool_errors=2, tokens=100)), + challenger=_per_seed( + ( + _metrics(tool_errors=1, tokens=100), + _metrics(tool_errors=1, tokens=100), + _metrics(tool_errors=1, tokens=101), + ) + ), + ) + + report = _evaluate(_runner(), replay) + + assert report.accepted is False + assert report.reason == WORSE_TOKENS + assert report.champion_metrics.tokens == 100 + assert report.challenger_metrics.tokens == 100 + + def test_a_gain_hidden_by_rounding_still_accepts(self) -> None: + """The trap in the other direction: 10.333… → 10.0 is a real gain. + + Nothing else moves, so an implementation comparing the rounded ints + sees 10 against 10 and calls it ``no_practical_gain``. + """ + replay = _replay( + champion=_per_seed( + ( + _metrics(call_count=10), + _metrics(call_count=10), + _metrics(call_count=11), + ) + ), + challenger=_flat(_metrics(call_count=10)), + ) + + report = _evaluate(_runner(), replay) + + assert report.accepted is True + assert report.reason == ACCEPTED + assert report.champion_metrics.call_count == 10 + assert report.challenger_metrics.call_count == 10 + + # -- isolation --------------------------------------------------------- + + def test_the_source_imports_no_runtime_or_mcp(self) -> None: + modules = _module_level_imports(ast.parse(_evaluate_source())) + + offenders = sorted( + name + for name in modules + if any( + name == prefix or name.startswith(f"{prefix}.") + for prefix in _FORBIDDEN_IMPORT_PREFIXES + ) + ) + + assert offenders == [] + + def test_the_gate_is_not_an_mcp_tool(self) -> None: + """Imported here rather than at module level: the leaf owes it nothing.""" + import molmcp + + for name in ("evaluate", "EvaluationReport", "ContractRunner", "Metrics"): + assert name not in molmcp.__all__ diff --git a/tests/test_gate.py b/tests/test_gate.py new file mode 100644 index 0000000..536a46c --- /dev/null +++ b/tests/test_gate.py @@ -0,0 +1,758 @@ +"""``run_gate`` — the wiring contract under one repository root. + +``molmcp gate`` is this repository's single required GitHub check, and what +it decides is narrow: whether three copies of one sentence still agree — the +literal ``run:`` of the pull-request job, the literal ``run:`` of the +schedule job, and the pre-commit hook's ``entry:``. It decides nothing else. +Lint and tests belong to ``ci.yml``'s OS/Python matrix; a gate that shelled +out to them would be a second, slower copy of that matrix, and a gate that +read the environment would decide differently on a laptop than on a runner. + +Every verdict test therefore hands ``run_gate`` a *root* and reads the +report. The two trees under ``tests/fixtures/gate/`` carry the same relative +paths production reads — ``.github/workflows/official-gate.yml`` and +``.pre-commit-config.yaml``. ``wired/`` is a legal wiring; ``contract-fail/`` +is that same wiring with one line changed, the hook's ``entry:`` wrapped in +``bash -c 'uv sync --extra dev && …'`` so that it no longer equals the PR +job's ``run:``. One planted breakage is what makes the reported failure +attributable to a line rather than to the tree. + +The static half states what ``gate.py`` must never grow. An earlier draft of +this spec had a ``--full`` profile that called spec 11's ``evaluate``; it was +deleted because an evaluation needs two subagents and a GitHub runner has +none, and because the module it named never existed. The constants, the +signature, and the report's two fields are pinned here so that the deleted +profile cannot walk back in through the stale acceptance file that still +mentions ``FULL_RUN``. + +``TestOfficialGateParity`` reads no fixture. It opens the files this repository +actually ships and asserts each copied token against ``gate.GATE_RUN`` — the +same equality ``run_gate`` checks, asserted from the other side. It is not a +diff between the workflow and the pre-commit config: two copies that drifted +together would still agree with each other and still be wrong, so each is +compared against the constant that is the authority. Its scanner is local for +the same reason. Borrowing ``gate.py``'s own reader would leave these +assertions blind to the one bug that would matter most — a reader that +mis-parses the repository's shape, and so compares nothing at all. +""" + +from __future__ import annotations + +import ast +import dataclasses +import inspect +import shutil +from pathlib import Path +from typing import NamedTuple + +import pytest +from _ast_checks import reads_environment + +from molmcp import gate + +#: Imported as a module, not by name: several tests below ask which names the +#: module *has* (``hasattr(gate, "FULL_RUN")``), which needs the module object +#: rather than a list of names that already resolved. +run_gate = gate.run_gate +GateReport = gate.GateReport + +_REPO = Path(__file__).resolve().parents[1] + +#: The module under test, read as data by the static half. +_GATE_SOURCE = _REPO / "src" / "molmcp" / "gate.py" + +_FIXTURES = Path(__file__).resolve().parent / "fixtures" / "gate" + +#: A legal wiring: the two files agree on the one literal. +WIRED = _FIXTURES / "wired" + +#: The same tree with the hook's ``entry:`` wrapped, and nothing else moved. +CONTRACT_FAIL = _FIXTURES / "contract-fail" + +#: The two paths ``run_gate`` reads, relative to the root it is given. +_WORKFLOW = ".github/workflows/official-gate.yml" +_PRE_COMMIT = ".pre-commit-config.yaml" +_CONTRACT_FILES = (_WORKFLOW, _PRE_COMMIT) + +#: ``gate.py`` is the authority; the YAML files are serialized copies. +_CONSTANTS = ( + ("CHECK_NAME", "official/gate"), + ("PR_JOB_ID", "official-gate"), + ("SCHEDULE_JOB_ID", "official-gate-schedule"), + ("GATE_RUN", "uv run molmcp gate"), +) + +#: Names of the deleted profile. ``release.yml`` already owns job id ``gate``. +_DELETED_CONSTANTS = ("FULL_RUN", "CHEAP_RUN", "GATE_PROFILE") + +#: Parameters a profile would need. ``root`` is the whole signature. +_DELETED_PARAMETERS = ("full", "evaluate", "skip", "profile") + +#: The report's fields, in order. +_REPORT_FIELDS = ("ok", "failed") + +#: Fragments of the one planted breakage. A verdict that does not name the +#: offending token leaves the reader with the same search the gate just did. +_OFFENCE_FRAGMENTS = ("entry", "bash -c") + +#: Runners ``run_gate`` must not become. Lint and tests stay in ``ci.yml``. +_FORBIDDEN_IMPORTS = ("subprocess", "pytest", "ruff") + +#: Call names that would mean the verdict spawned a process. +_SPAWN_CALLS = frozenset({"Popen", "check_output", "check_call", "system", "run_safe"}) + + +def _gate_tree() -> ast.Module: + """``gate.py`` parsed, or a readable failure instead of an ``OSError``.""" + assert _GATE_SOURCE.is_file(), ( + f"{_GATE_SOURCE.relative_to(_REPO)} does not exist. The verdict has " + f"one owner: cli.py only dispatches to run_gate." + ) + return ast.parse(_GATE_SOURCE.read_text(encoding="utf-8")) + + +def _imported_modules(tree: ast.AST) -> set[str]: + """Every module name an ``import`` or ``from … import`` names.""" + modules: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + modules.update(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + module = node.module or "" + modules.add(module) + modules.update(f"{module}.{alias.name}" for alias in node.names) + return modules + + +def _called_names(tree: ast.AST) -> set[str]: + """Every simple name or attribute that appears in call position.""" + names: set[str] = set() + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + func = node.func + if isinstance(func, ast.Name): + names.add(func.id) + elif isinstance(func, ast.Attribute): + names.add(func.attr) + return names + + +def _wired_missing(tmp_path: Path, relative: str) -> Path: + """A copy of the wired tree with one of the two contract files removed.""" + root = tmp_path / "root" + shutil.copytree(WIRED, root) + (root / relative).unlink() + return root + + +def _messages(report: GateReport) -> str: + return "\n".join(report.failed) + + +class TestRunGate: + # -- the legal wiring ---------------------------------------------- + + def test_wired_fixture_is_ok(self) -> None: + assert run_gate(root=WIRED).ok is True + + def test_wired_fixture_reports_no_failure(self) -> None: + report = run_gate(root=WIRED) + + assert report.failed == () + + def test_returns_a_gate_report(self) -> None: + assert isinstance(run_gate(root=WIRED), GateReport) + + # -- the planted breakage ------------------------------------------ + + def test_contract_fail_fixture_is_not_ok(self) -> None: + assert run_gate(root=CONTRACT_FAIL).ok is False + + def test_contract_fail_fixture_reports_a_failure(self) -> None: + report = run_gate(root=CONTRACT_FAIL) + + assert report.failed != () + + @pytest.mark.parametrize("fragment", _OFFENCE_FRAGMENTS) + def test_contract_fail_verdict_names_the_disagreeing_token( + self, fragment: str + ) -> None: + """The one changed line is the hook's wrapped ``entry:``.""" + report = run_gate(root=CONTRACT_FAIL) + + assert fragment in _messages(report), ( + f"the verdict does not name {fragment!r}: {report.failed}" + ) + + def test_failed_is_a_tuple_of_strings(self) -> None: + report = run_gate(root=CONTRACT_FAIL) + + assert isinstance(report.failed, tuple) + assert all(isinstance(message, str) for message in report.failed) + + # -- a root that is missing half the contract ----------------------- + + @pytest.mark.parametrize("relative", _CONTRACT_FILES) + def test_missing_contract_file_is_a_verdict_not_an_exception( + self, tmp_path: Path, relative: str + ) -> None: + """A half-wired tree is red, not a traceback out of the gate.""" + root = _wired_missing(tmp_path, relative) + + assert run_gate(root=root).ok is False + + @pytest.mark.parametrize("relative", _CONTRACT_FILES) + def test_missing_contract_file_reports_a_failure( + self, tmp_path: Path, relative: str + ) -> None: + root = _wired_missing(tmp_path, relative) + + assert run_gate(root=root).failed != () + + def test_empty_root_is_not_ok(self, tmp_path: Path) -> None: + assert run_gate(root=tmp_path).ok is False + + # -- signature ------------------------------------------------------ + + def test_signature_is_root_and_nothing_else(self) -> None: + assert list(inspect.signature(run_gate).parameters) == ["root"] + + def test_root_is_keyword_only(self) -> None: + parameter = inspect.signature(run_gate).parameters["root"] + + assert parameter.kind is inspect.Parameter.KEYWORD_ONLY + + def test_root_has_no_default(self) -> None: + """The CLI passes ``Path.cwd()``; the gate never guesses a root.""" + parameter = inspect.signature(run_gate).parameters["root"] + + assert parameter.default is inspect.Parameter.empty + + @pytest.mark.parametrize("name", _DELETED_PARAMETERS) + def test_carries_no_profile_parameter(self, name: str) -> None: + """One profile. There is no agent in a runner to evaluate with.""" + assert name not in inspect.signature(run_gate).parameters + + # -- the report ----------------------------------------------------- + + def test_report_field_names_are_ok_and_failed(self) -> None: + names = tuple(field.name for field in dataclasses.fields(GateReport)) + + assert names == _REPORT_FIELDS + + def test_report_has_exactly_two_fields(self) -> None: + assert len(dataclasses.fields(GateReport)) == 2 + + @pytest.mark.parametrize("field_name", _REPORT_FIELDS) + def test_report_is_frozen(self, field_name: str) -> None: + report = run_gate(root=WIRED) + + with pytest.raises(dataclasses.FrozenInstanceError): + setattr(report, field_name, "mutated") + + def test_report_uses_slots(self) -> None: + report = run_gate(root=WIRED) + + assert hasattr(GateReport, "__slots__") + assert not hasattr(report, "__dict__") + + def test_ok_is_a_bool(self) -> None: + assert isinstance(run_gate(root=WIRED).ok, bool) + + # -- constants ------------------------------------------------------ + + @pytest.mark.parametrize(("name", "literal"), _CONSTANTS) + def test_constant_equals_its_literal(self, name: str, literal: str) -> None: + assert getattr(gate, name) == literal + + def test_pr_job_id_is_not_the_release_gate(self) -> None: + """``.github/workflows/release.yml`` already owns job id ``gate``.""" + assert gate.PR_JOB_ID != "gate" + + def test_the_two_jobs_have_different_ids(self) -> None: + assert gate.SCHEDULE_JOB_ID != gate.PR_JOB_ID + + def test_check_name_is_a_job_name_not_a_job_id(self) -> None: + """GitHub matches a required check on the job's ``name:``.""" + assert gate.CHECK_NAME != gate.PR_JOB_ID + + @pytest.mark.parametrize("name", _DELETED_CONSTANTS) + def test_module_carries_no_second_profile_literal(self, name: str) -> None: + """A second literal is a second thing for parity to disagree with.""" + assert not hasattr(gate, name) + + # -- what the source must never grow, read off the source ----------- + + def test_source_never_reads_the_environment(self) -> None: + """A gate configured by the environment decides two things at once.""" + assert not reads_environment(_gate_tree()) + + @pytest.mark.parametrize("module", _FORBIDDEN_IMPORTS) + def test_source_imports_no_runner(self, module: str) -> None: + """Lint and tests are ``ci.yml``'s matrix; the gate checks wiring.""" + imported = _imported_modules(_gate_tree()) + offenders = { + name for name in imported if name == module or name.startswith(f"{module}.") + } + + assert offenders == set() + + def test_source_imports_nothing_named_evaluate(self) -> None: + """Evaluation needs two subagents; a GitHub runner has none.""" + imported = _imported_modules(_gate_tree()) + + assert [name for name in imported if "evaluate" in name] == [] + + def test_source_spawns_no_process(self) -> None: + called = _called_names(_gate_tree()) + + assert called & _SPAWN_CALLS == set() + + +# -- the repository's own copies ---------------------------------------- + +#: The two files this repository ships, at the same relative paths the +#: fixtures use. They do not exist until the workflow and the hook are +#: written, which is what every message below has to survive readably. +_REPO_WORKFLOW = _REPO / _WORKFLOW +_REPO_PRE_COMMIT = _REPO / _PRE_COMMIT + +#: The product matrix. This spec does not fold the gate into it. +_CI_WORKFLOW = _REPO / ".github" / "workflows" / "ci.yml" + +#: The release gate, which already owns the job id ``gate``. +_RELEASE_WORKFLOW = _REPO / ".github" / "workflows" / "release.yml" + +#: Both project files carry the same frontmatter, and ``ci.config`` in it +#: still points at the product matrix. +_PROJECT_DOCS = (_REPO / "CLAUDE.md", _REPO / "AGENTS.md") +_CI_CONFIG = ".github/workflows/ci.yml" + +_WHY_WORKFLOW = ( + f"Nothing runs {gate.GATE_RUN!r} on a pull request, so the " + f"{gate.CHECK_NAME!r} required check reports nothing and a branch " + f"protected by it is protected by an absence." +) + +_WHY_PRE_COMMIT = ( + f"Nothing runs {gate.GATE_RUN!r} before a push, so a broken wiring is " + f"first heard about from GitHub." +) + +_WHY_REPO_FILE = ( + "This spec does not create or move it; it is read here only to show that " + "it stayed where it was." +) + +#: The hook carrying the literal. The same word as ``PR_JOB_ID`` on purpose: +#: one check, one name in every file that mentions it. +_HOOK_ID = "official-gate" + +#: The hook the commit stage keeps, and the stage names pre-commit uses. +_COMMIT_HOOK_ID = "ci-lint" +_COMMIT_STAGE = "pre-commit" +_PUSH_STAGE = "pre-push" + +#: The two keys pair 2 pins: the pull-request job's gate ``run:`` and the +#: hook's ``entry:``. Both are read against ``GATE_RUN``, never against each +#: other. +_PAIR_TWO = ("run", "entry") + +#: Wrappers that would make a token a different string from the one the other +#: file runs. ``uv sync --extra dev`` is a prior Install step, not the token. +_WRAPPERS = ("uv sync", "bash -c") + +#: How the gate step is picked out before its literal is read. Not a second +#: call literal: it selects which ``run:`` to compare, and the comparison is +#: always against ``gate.GATE_RUN``. +_GATE_CALL = "molmcp gate" + +#: The two jobs, read off the authority. +_JOB_IDS = (gate.PR_JOB_ID, gate.SCHEDULE_JOB_ID) + +#: A GitHub expression is legal in ``if:`` and ``concurrency:`` and forbidden +#: in a ``run:``: what it expands to on a runner is not what was compared. +_EXPRESSION = "${{" + +#: Enough of ``ci.yml``'s matrix to show it is still the product matrix. +_MATRIX_TOKENS = ("matrix:", "os:", "python-version:") + +#: YAML's block scalar indicators. +_BLOCK_SCALARS = frozenset({"|", "|-", "|+", ">", ">-", ">+"}) + + +class _Entry(NamedTuple): + """One significant line of a scanned file. + + Attributes: + number: 1-based line number, so a failure can name a location. + indent: Leading spaces, which is what nesting means in these files. + text: The line with surrounding whitespace removed. + """ + + number: int + indent: int + text: str + + +def _rel(path: Path) -> str: + return path.relative_to(_REPO).as_posix() + + +def _read(path: Path, why: str) -> str: + """The file's text, or a readable failure instead of an ``OSError``.""" + assert path.is_file(), f"{_rel(path)} does not exist. {why}" + return path.read_text(encoding="utf-8") + + +def _scan(text: str) -> tuple[_Entry, ...]: + """*text* as significant lines: blanks and whole-line comments dropped.""" + entries: list[_Entry] = [] + for number, raw in enumerate(text.splitlines(), 1): + stripped = raw.strip() + if not stripped or stripped.startswith("#"): + continue + entries.append(_Entry(number, len(raw) - len(raw.lstrip(" ")), stripped)) + return tuple(entries) + + +def _under(entries: tuple[_Entry, ...], index: int) -> tuple[_Entry, ...]: + """Every line nested under ``entries[index]``.""" + parent = entries[index].indent + end = index + 1 + while end < len(entries) and entries[end].indent > parent: + end += 1 + return entries[index + 1 : end] + + +def _unquoted(value: str) -> str: + """*value* without one matching pair of surrounding quotes.""" + if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'": + return value[1:-1] + return value + + +def _pair(entry: _Entry) -> tuple[str, str] | None: + """*entry* as a ``key: value`` mapping entry, or ``None``. + + A leading ``- `` is dropped, so the first key of a list item reads like + any other key. A key holding a space is not a key: that is a line of + shell inside a block scalar. + """ + text = entry.text[2:].lstrip() if entry.text.startswith("- ") else entry.text + key, separator, value = text.partition(":") + if not separator or not key or " " in key: + return None + return key, _unquoted(value.strip()) + + +def _scalar(block: tuple[_Entry, ...], key: str) -> str | None: + """*key* read off the direct children of *block*, or ``None``.""" + if not block: + return None + depth = min(entry.indent for entry in block) + for entry in block: + if entry.indent != depth: + continue + found = _pair(entry) + if found is not None and found[0] == key: + return found[1] + return None + + +def _items(block: tuple[_Entry, ...], key: str) -> tuple[str, ...]: + """*key* read off *block* as a list, written flow (``[a, b]``) or nested.""" + if not block: + return () + depth = min(entry.indent for entry in block) + for index, entry in enumerate(block): + if entry.indent != depth: + continue + found = _pair(entry) + if found is None or found[0] != key: + continue + value = found[1] + if value.startswith("[") and value.endswith("]"): + inner = value[1:-1].strip() + if not inner: + return () + return tuple(_unquoted(part.strip()) for part in inner.split(",")) + if value: + return (value,) + return tuple( + _unquoted(child.text[2:].strip()) + for child in _under(block, index) + if child.text.startswith("- ") + ) + return () + + +def _runs(block: tuple[_Entry, ...]) -> tuple[str, ...]: + """Every ``run:`` anywhere in a job, a block scalar joined into one line.""" + texts: list[str] = [] + for index, entry in enumerate(block): + found = _pair(entry) + if found is None or found[0] != "run": + continue + value = found[1] + if value and value not in _BLOCK_SCALARS: + texts.append(value) + else: + texts.append(" ".join(child.text for child in _under(block, index))) + return tuple(texts) + + +def _jobs(path: Path, why: str) -> dict[str, tuple[_Entry, ...]]: + """Every job of the workflow at *path*, by id.""" + entries = _scan(_read(path, why)) + top: tuple[_Entry, ...] = () + for index, entry in enumerate(entries): + if entry.indent == 0 and _pair(entry) == ("jobs", ""): + top = _under(entries, index) + break + assert top, f"{_rel(path)} has no top-level `jobs:` mapping." + depth = min(entry.indent for entry in top) + jobs: dict[str, tuple[_Entry, ...]] = {} + for index, entry in enumerate(top): + if entry.indent != depth: + continue + found = _pair(entry) + if found is not None: + jobs[found[0]] = _under(top, index) + return jobs + + +def _gate_jobs() -> dict[str, tuple[_Entry, ...]]: + return _jobs(_REPO_WORKFLOW, _WHY_WORKFLOW) + + +def _job(job_id: str) -> tuple[_Entry, ...]: + """The job *job_id*, or a failure naming the ids that are there.""" + jobs = _gate_jobs() + assert job_id in jobs, ( + f"{_WORKFLOW} has no job with id {job_id!r}; ids found: " + f"{sorted(jobs) or 'none'}. The pull-request job reports the check " + f"and the scheduled job re-checks the wiring on a timer." + ) + return jobs[job_id] + + +def _gate_run(job_id: str) -> str: + """The one ``run:`` of *job_id* that calls the gate.""" + calls = [text for text in _runs(_job(job_id)) if _GATE_CALL in text] + assert len(calls) == 1, ( + f"{_WORKFLOW}: job {job_id!r} has {len(calls)} step(s) whose run: " + f"mentions {_GATE_CALL!r}, and exactly one of them is the gate call. " + f"`uv sync --extra dev` is the prior Install step, not the compared " + f"token. Found: {calls}." + ) + return calls[0] + + +def _hooks() -> dict[str, tuple[_Entry, ...]]: + """Every pre-commit hook, by id.""" + entries = _scan(_read(_REPO_PRE_COMMIT, _WHY_PRE_COMMIT)) + hooks: dict[str, tuple[_Entry, ...]] = {} + for index, entry in enumerate(entries): + if not entry.text.startswith("- "): + continue + found = _pair(entry) + if found is not None and found[0] == "id": + hooks[found[1]] = _under(entries, index) + return hooks + + +def _hook(hook_id: str) -> tuple[_Entry, ...]: + """The hook *hook_id*, or a failure naming the ids that are there.""" + hooks = _hooks() + assert hook_id in hooks, ( + f"{_PRE_COMMIT} has no hook with id {hook_id!r}; ids found: {sorted(hooks)}." + ) + return hooks[hook_id] + + +def _hook_entry() -> str: + """The gate hook's ``entry:``, which is one half of pair 2.""" + entry = _scalar(_hook(_HOOK_ID), "entry") + assert entry is not None, ( + f"{_PRE_COMMIT}: hook {_HOOK_ID!r} has no entry:; expected " + f"entry: {gate.GATE_RUN}." + ) + return entry + + +def _token(key: str) -> str: + """One of pair 2's two tokens, named by the key that carries it.""" + return _gate_run(gate.PR_JOB_ID) if key == "run" else _hook_entry() + + +def _frontmatter(path: Path) -> tuple[_Entry, ...]: + """The lines between the opening and closing ``---`` fences.""" + lines = _read(path, _WHY_REPO_FILE).splitlines() + assert lines[:1] == ["---"], ( + f"{_rel(path)} must open with a --- frontmatter fence; its first line " + f"is {lines[:1]!r}." + ) + closing = next( + (index for index, line in enumerate(lines[1:], 1) if line.strip() == "---"), + None, + ) + assert closing is not None, ( + f"{_rel(path)} opens a --- frontmatter fence that is never closed." + ) + return _scan("\n".join(lines[1:closing])) + + +def _ci_config(path: Path) -> str | None: + """``mol_project.ci.config`` of *path*'s frontmatter.""" + entries = _frontmatter(path) + for index, entry in enumerate(entries): + if _pair(entry) == ("ci", ""): + return _scalar(_under(entries, index), "config") + return None + + +class TestOfficialGateParity: + # -- pair 2: the PR job's run: and the hook's entry: ---------------- + + @pytest.mark.parametrize("key", _PAIR_TWO) + def test_pair_two_token_is_the_one_literal(self, key: str) -> None: + """Each copy against the constant, never against the other copy.""" + token = _token(key) + + assert token == gate.GATE_RUN, ( + f"the {key}: token is {token!r}, not {gate.GATE_RUN!r}. Local, " + f"pull request and timer must run the same sentence, character " + f"for character; gate.GATE_RUN is the authority and both files " + f"are copies of it." + ) + + @pytest.mark.parametrize("wrapper", _WRAPPERS) + @pytest.mark.parametrize("key", _PAIR_TWO) + def test_pair_two_token_is_not_wrapped(self, key: str, wrapper: str) -> None: + token = _token(key) + + assert wrapper not in token, ( + f"the {key}: token {token!r} wraps the call in {wrapper!r}, which " + f"makes it a different string from the one the other file runs. " + f"Installing is a prior step, not part of the compared token." + ) + + # -- the check name, and the job ids --------------------------------- + + def test_workflow_defines_exactly_the_two_jobs(self) -> None: + assert sorted(_gate_jobs()) == sorted(_JOB_IDS) + + def test_no_job_takes_the_release_gate_id(self) -> None: + """``release.yml`` owns ``gate``; two jobs under one id is a rename.""" + assert "gate" not in _gate_jobs() + + def test_pull_request_job_is_named_the_required_check(self) -> None: + name = _scalar(_job(gate.PR_JOB_ID), "name") + + assert name == gate.CHECK_NAME == "official/gate", ( + f"{_WORKFLOW}: job {gate.PR_JOB_ID!r} has name: {name!r}. GitHub " + f"matches a required check on the name it displays, not on the " + f"job id, so this one line is what makes the check exist." + ) + + def test_schedule_job_is_not_named_the_required_check(self) -> None: + name = _scalar(_job(gate.SCHEDULE_JOB_ID), "name") + + assert name != gate.CHECK_NAME, ( + f"{_WORKFLOW}: job {gate.SCHEDULE_JOB_ID!r} is also named " + f"{gate.CHECK_NAME!r}, which would let a timer report the check a " + f"pull request is supposed to report." + ) + + def test_schedule_job_runs_the_same_literal(self) -> None: + assert _gate_run(gate.SCHEDULE_JOB_ID) == gate.GATE_RUN + + # -- literal run:, and nothing from the environment ------------------ + + @pytest.mark.parametrize("job_id", _JOB_IDS) + def test_no_run_expands_an_expression(self, job_id: str) -> None: + expanded = [text for text in _runs(_job(job_id)) if _EXPRESSION in text] + + assert expanded == [], ( + f"{_WORKFLOW}: job {job_id!r} has {len(expanded)} run: holding " + f"{_EXPRESSION!r}, first {expanded[:1]!r}. `if:` may hold an " + f"expression; a run: may not, because what it expands to on a " + f"runner is not what parity compared." + ) + + @pytest.mark.parametrize("job_id", _JOB_IDS) + def test_job_selects_nothing_from_the_environment(self, job_id: str) -> None: + lines = [ + entry.number + for entry in _job(job_id) + if (found := _pair(entry)) is not None and found[0] == "env" + ] + + assert lines == [], ( + f"{_WORKFLOW}: job {job_id!r} has env: on line(s) {lines}. There " + f"is one profile, so an env: here can only be selecting a second " + f"one, and the gate would then decide two different things." + ) + + # -- which stage the hook runs in ------------------------------------ + + def test_gate_hook_runs_only_before_a_push(self) -> None: + stages = _items(_hook(_HOOK_ID), "stages") + + assert stages == (_PUSH_STAGE,), ( + f"{_PRE_COMMIT}: hook {_HOOK_ID!r} has stages: {list(stages)}, not " + f"[{_PUSH_STAGE}]. The gate runs before a push; the commit stage " + f"stays fast." + ) + + def test_commit_stage_still_holds_ci_lint(self) -> None: + stages = _items(_hook(_COMMIT_HOOK_ID), "stages") + + assert _COMMIT_STAGE in stages, ( + f"{_PRE_COMMIT}: hook {_COMMIT_HOOK_ID!r} no longer lists the " + f"{_COMMIT_STAGE!r} stage; it is what that stage holds." + ) + + def test_commit_stage_does_not_hold_the_gate(self) -> None: + assert _COMMIT_STAGE not in _items(_hook(_HOOK_ID), "stages") + + # -- what this spec leaves where it found it ------------------------- + + @pytest.mark.parametrize("token", _MATRIX_TOKENS) + def test_ci_workflow_still_carries_the_product_matrix(self, token: str) -> None: + text = _read(_CI_WORKFLOW, _WHY_REPO_FILE) + + assert token in text, ( + f"{_rel(_CI_WORKFLOW)} no longer mentions {token!r}. Lint and " + f"tests stay on the OS/Python matrix; the gate checks wiring and " + f"replaces none of it." + ) + + def test_ci_workflow_does_not_run_the_gate(self) -> None: + text = _read(_CI_WORKFLOW, _WHY_REPO_FILE) + + assert _GATE_CALL not in text, ( + f"{_rel(_CI_WORKFLOW)} runs {_GATE_CALL!r}. The required check is " + f"one job in one file; running it across a matrix reports the same " + f"verdict six times under six names." + ) + + def test_release_workflow_keeps_its_gate_job(self) -> None: + jobs = _jobs(_RELEASE_WORKFLOW, _WHY_REPO_FILE) + + assert "gate" in jobs, ( + f"{_rel(_RELEASE_WORKFLOW)} no longer has job id 'gate'; that job " + f"is why the new one is called {gate.PR_JOB_ID!r}." + ) + + @pytest.mark.parametrize("path", _PROJECT_DOCS, ids=lambda p: p.name) + def test_project_doc_still_points_ci_config_at_the_matrix(self, path: Path) -> None: + configured = _ci_config(path) + + assert configured == _CI_CONFIG, ( + f"{_rel(path)} frontmatter has mol_project.ci.config " + f"{configured!r}. It names the product matrix, and this spec adds " + f"a required check beside it rather than moving it." + ) diff --git a/tests/test_harness.py b/tests/test_harness.py new file mode 100644 index 0000000..b661d53 --- /dev/null +++ b/tests/test_harness.py @@ -0,0 +1,1491 @@ +"""Mirrors ``src/molmcp/harness.py`` — the fold that serves N harness sources. + +The first line names the mirrored module on purpose. Four modules in this +directory already begin ``test_harness_`` and none of them mirrors anything +under ``src/``: ``test_harness_agents.py``, ``test_harness_cases.py`` and +``test_harness_eval.py`` hold ``scripts/`` and its agent files to their +disciplines, and ``test_harness_catalog_fixture.py`` parses ``docs/``. This +one is the ``src/molmcp/harness.py`` mirror the layout rule asks for, and +covers only the symbols that module owns. + +Seven units are exercised here, each in isolation: + +*``pointer_path`` is a security fix, not a formatting helper.* +``HarnessSource.name`` is governed only as "non-empty, whitespace-free": +``settings.py:152-159`` puts the ``/`` and ``@`` rejection in an ``elif`` +that explicitly excludes ``name``, and the class docstring says why — an +operator who may name an index source ``MolCrafts`` may name a harness +source ``MolCrafts``. So ``HarnessSource(name="../../evil")`` constructs +today, and the moment a name is interpolated into ``harness.{name}.pointer`` +it becomes path *structure* rather than a label. The guard is at the point of +use because that is the only place that knows the name is about to be a path +segment. + +*``assert_servable`` is the strict end of a parsed locator.* A GitHub +locator is servable without a path, including ``MolCrafts/harness@main``. +A local locator must already be an absolute or ``~/`` path — relative +spellings are refused at ``HarnessSource`` construction, not here — and +must name a checkout (``.git`` exists). ``enable=()`` is not a filter in +this slice. ``HARNESS_COORDINATES`` is gone. + +*``SourcedComponent`` pairs an origin with an untouched spec.* +``components/models.py:120-127`` pins ``id == f"{kind}.{name}"`` and +``_MEMBER_PATTERN`` admits nothing else, so ``official.provider.demo`` is not +a constructible id. The cross-source key is the ``(source, spec)`` pair, in +this layer — the arrangement ``tests/test_no_builtin_harness_source.py`` +already spells out in its failure message. + +*``fold_components`` is first-wins, and reports the loser.* The catalogs are +read from real ``harness.toml`` files written under ``tmp_path``: nothing here +patches ``load_harness_catalog``, because a suite that fakes the reader it +depends on proves only the call order (``notes.md:faked-seam-hides-broken-reader``). +Nothing fetches, no store is constructed, and no pointer is bound — how a +catalog reaches a checkout is ``activated_checkouts``' problem, and is covered +by the class below. + +*``ComponentFold`` carries one base per source, as the authored string.* +``root_for`` is the only place a tree and a catalog's ``component_root`` are +joined, and it joins them against *that source's own* ``Checkout.tree`` — so a +base belonging to the wrong tree is not a state the type can hold. The five +disagreeing constructions its ``__post_init__`` refuses are built by hand, +because four of them cannot be reached through ``fold_components`` at all. + +*``checkout_planes`` is the provider half of one property.* The failure the +``component_root`` key exists to make unreachable is being applied in one arm +and forgotten in the other — providers resolving while overlays do not, an +install that *looks* like it works. This file owns the provider arm, driven +over a real tree; ``tests/test_stack.py`` owns the overlay arm, because that +one is ``create_stack``'s composition rather than this module's. + +*``activated_checkouts`` is driven with no seam at all.* The five names +``tests/test_stack.py``'s ``_wire`` fakes — ``Activation``, +``ImmutableGitStore``, ``GitHubTransport``, ``load_harness_catalog`` and +``WorkerProvider`` — are the five this file never patches. That is the same +rule again, and it is the reason this class exists rather than one more +``_wire`` test: a seam proves the composition *order* and nothing whatsoever +about the functions it replaces. The store is planted by hand under +``tmp_path`` and the pointers are literal version-1 JSON, so a real +:class:`~molmcp.components.ImmutableGitStore` and a real +:meth:`~molmcp.components.Activation.bind` do the work. +""" + +from __future__ import annotations + +import dataclasses +import json +import logging +import os +import subprocess +from collections.abc import Sequence +from pathlib import Path + +import pytest + +from molmcp import harness +from molmcp.components import CatalogError, ComponentKind, ComponentSpec +from molmcp.components.activate import _POINTER_KEYS, ACTIVATION_VERSION +from molmcp.components.locator import LocatorError +from molmcp.config import AppConfig, ConfigurationError +from molmcp.settings import HarnessSource + +#: Frozen *and* slotted dataclasses answer a rebound field with either error, +#: depending on which guard fires first; the idiom is +#: ``tests/test_components/test_activate.py:74``. +_ASSIGN_ERRORS = (AttributeError, dataclasses.FrozenInstanceError) + +#: Two distinct 40-character lowercase hex SHAs — the only shape +#: ``HarnessCatalog`` accepts as identity (``models.py:57``). +_OFFICIAL_SHA = "0123456789abcdef0123456789abcdef01234567" +_PRIVATE_SHA = "89abcdef0123456789abcdef0123456789abcdef" + +#: The module logger the fold reports a displaced component through. Named +#: here rather than derived, so a rename has to pass through this file. +_LOGGER = "molmcp.harness" + + +def _provider(name: str, entrypoint: str) -> ComponentSpec: + """One provider row. Its ``id`` is ``provider.`` and cannot be else.""" + return ComponentSpec( + kind=ComponentKind.PROVIDER, + name=name, + id=f"provider.{name}", + path=f"providers/{name}/plane.py", + entrypoint=entrypoint, + ) + + +def _skill(name: str) -> ComponentSpec: + """One skill row, used only to prove the fold keeps to the asked-for kind.""" + return ComponentSpec( + kind=ComponentKind.SKILL, + name=name, + id=f"skill.{name}", + path=f"skills/{name}.md", + ) + + +def _catalog_toml(specs: Sequence[ComponentSpec], *, component_root: str = "") -> str: + """Render specs as the ``harness.toml`` a real checkout would carry. + + Every catalog must declare a ``daily`` and a ``dev`` bundle + (``catalog.py:87-88``) and bundle members must be non-empty and resolve + (``models.py:167``), so both bundles list every component in the file. + No catalog-level ``requires`` is emitted: eligibility is + ``load_harness_catalog``'s subject, not the fold's. + + Args: + specs: Component rows, rendered in the order they are given — which + is the catalog order the fold preserves within a source. + component_root: Optional top-level ``component_root``. It is emitted + **above** the first ``[[component]]``, because a bare key written + after a table header belongs to that table and TOML would read it + as a component field. The empty default emits no key at all, + which is what every catalog in this file carried before the key + existed and what every rootless catalog carries now. + + Returns: + The whole document, component paths exactly as the specs authored + them: ``component_root`` is carried beside them and never folded in. + """ + members = ", ".join(f'"{spec.id}"' for spec in specs) + rows: list[str] = [] + for spec in specs: + row = [ + "[[component]]", + f'kind = "{spec.kind.value}"', + f'name = "{spec.name}"', + f'path = "{spec.path}"', + ] + if spec.entrypoint is not None: + row.append(f'entrypoint = "{spec.entrypoint}"') + rows.append("\n".join(row)) + for bundle in ("daily", "dev"): + rows.append( + f'[[component]]\nkind = "bundle"\nname = "{bundle}"\nmembers = [{members}]' + ) + document = "\n\n".join(rows) + "\n" + if not component_root: + return document + return f'component_root = "{component_root}"\n\n' + document + + +def _checkout( + root: Path, + source: str, + sha: str, + specs: Sequence[ComponentSpec], + *, + component_root: str = "", +) -> harness.Checkout: + """A checkout whose tree really holds the catalog these specs describe. + + *component_root* goes into that catalog, never into the tree. + ``Checkout.tree`` means "where ``harness.toml`` sits" and keeps that + contract whatever the value is: folding the component root into the tree + would move the catalog file too. + """ + tree = root / source / "tree" + tree.mkdir(parents=True) + (tree / "harness.toml").write_text( + _catalog_toml(specs, component_root=component_root), encoding="utf-8" + ) + return harness.Checkout(sha=sha, tree=tree, source=source, enable=None) + + +def _warnings(caplog: pytest.LogCaptureFixture) -> list[logging.LogRecord]: + """Only this module's warnings; a neighbour's INFO is not the report.""" + return [ + record + for record in caplog.records + if record.name == _LOGGER and record.levelno == logging.WARNING + ] + + +def _entries(root: Path) -> list[Path]: + """Everything that exists under *root*, for a before/after comparison.""" + return sorted(root.rglob("*")) + + +def _source( + name: str, + *, + locator: str | None = None, +) -> HarnessSource: + """One complete ``harness`` entry, the shape ``_harness_locator`` hands over. + + The locator is filled in because a real one always is by the time + this function sees it, and is otherwise irrelevant: ``activated_checkouts`` + reads the pointer, never the repository. + """ + return HarnessSource(name=name, locator=locator or "molcrafts/harness") + + +def _config_and_root(tmp_path: Path) -> tuple[AppConfig, Path]: + """A resolved config, and the cache root its store and pointers hang off. + + The root is read back off the config rather than recomputed from + *tmp_path*: ``AppConfig.from_dict`` resolves the path, and on darwin + ``/var`` is a symlink to ``/private/var``, so the two spellings are not + the same string. + """ + config = AppConfig.from_dict( + {"schema_version": "2", "cache_dir": str(tmp_path / "cache")}, + workspace_root=tmp_path, + ) + assert config.cache_dir is not None + return config, config.cache_dir + + +def _publish_by_hand( + store_root: Path, + sha: str, + *, + owner: str = "molcrafts", + repo: str = "harness", +) -> Path: + """Plant one complete SHA directory the way the store reads it back. + + ``/commits//`` holding ``metadata.json`` and ``tree/`` is + the layout ``components/store.py:43-52`` documents and the exact pair + ``ImmutableGitStore.has`` checks at ``store.py:90-91``. It is written here + rather than fetched: ``publish`` is the only path that reaches the network, + and no test in this file calls it. + + Returns: + The flattened catalog root — what ``tree_path(sha)`` will answer. + """ + sha_dir = store_root / "commits" / sha + tree = sha_dir / "tree" + tree.mkdir(parents=True) + (sha_dir / "metadata.json").write_text( + json.dumps({"sha": sha, "owner": owner, "repo": repo}), encoding="utf-8" + ) + return tree + + +def _pointer_payload(active: str | None) -> dict[str, object]: + """A version-1 activation record, keyed exactly as ``_POINTER_KEYS``.""" + return { + "version": ACTIVATION_VERSION, + "active": active, + "staging": None, + "previous": None, + } + + +def _write_pointer(path: Path, active: str | None) -> None: + """Write one activation pointer file, JSON literal, no ``Activation``. + + Writing the file by hand is the point: ``stage`` and ``promote`` are the + only writers in the product and neither has a production caller, so a test + that reached for them would be exercising a path no install runs. + """ + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(_pointer_payload(active)), encoding="utf-8") + + +class TestPointerPath: + """One pointer file per source, and a name that can only be a label. + + The guard's reserved set is exactly ``{".", ".."}`` — kept for symmetry + with :data:`molmcp.components.store._RESERVED_SHA_KEYS`, **not** because + either one traverses. Embedded as ``harness.{name}.pointer`` neither is a + path segment at all: ``harness....pointer`` is one ordinary filename. The + separator, absolute-path and empty checks are the ones doing the real + work, and :meth:`test_the_dot_names_are_symmetry_and_the_separators_are_the_hole` + pins that difference so nobody later "simplifies" the guard by dropping + the half that matters. + """ + + def test_names_the_pointer_file_beside_the_store(self, tmp_path: Path) -> None: + """``/harness..pointer`` — a sibling of the store root.""" + root = tmp_path / "cache" + assert harness.pointer_path(root, "official") == ( + root / "harness.official.pointer" + ) + + def test_the_pointer_stays_a_direct_child_of_the_root(self, tmp_path: Path) -> None: + """One path segment, under the root it was handed, always.""" + root = tmp_path / "cache" + result = harness.pointer_path(root, "official") + assert result.parent == root + assert result.name == "harness.official.pointer" + + def test_two_names_never_map_to_one_file(self, tmp_path: Path) -> None: + """Distinct sources own distinct pointers, or activation is shared.""" + root = tmp_path / "cache" + official = harness.pointer_path(root, "official") + private = harness.pointer_path(root, "private") + assert official != private + assert {official.parent, private.parent} == {root} + + @pytest.mark.parametrize( + "name", + [ + pytest.param("..", id="reserved-dotdot"), + pytest.param(".", id="reserved-dot"), + pytest.param("../../evil", id="dotdot-escape"), + pytest.param("../../../evil", id="dotdot-escape-deeper"), + pytest.param("a/b", id="posix-separator"), + pytest.param("a\\b", id="windows-separator"), + pytest.param("/etc/passwd", id="absolute"), + pytest.param("", id="empty"), + ], + ) + def test_refuses_a_name_that_cannot_be_one_path_segment( + self, tmp_path: Path, name: str + ) -> None: + """Every hostile name is refused, and nothing lands on disk. + + The assertion that actually proves the guard is not the exception + type — it is that the filesystem is untouched, at the root and at the + place the naive ``/harness.{name}.pointer`` would have written. + A guard that raised *after* creating the parent directory would pass + an exception-only test. + """ + root = tmp_path / "cache" / "molmcp" + root.mkdir(parents=True) + before = _entries(tmp_path) + + with pytest.raises(ConfigurationError) as excinfo: + harness.pointer_path(root, name) + + message = str(excinfo.value) + # ``{name!r}`` is this repo's register for naming a rejected value + # (``store.py:180``, ``models.py:124``, ``settings.py:152``), and the + # only one that can name the empty string at all. + assert repr(name) in message + assert _entries(tmp_path) == before + naive = Path(os.path.normpath(root / f"harness.{name}.pointer")) + assert not naive.exists() + + def test_the_dot_names_are_symmetry_and_the_separators_are_the_hole( + self, tmp_path: Path + ) -> None: + """Which refusals are load-bearing, stated as an assertion. + + ``.`` and ``..`` interpolate into an ordinary filename that stays + inside the root; a separator or a deep ``..`` is what turns the name + into structure. Both are refused, but only the second group closes a + hole — this is the fact a later "simplification" would delete. + """ + root = tmp_path / "cache" / "molmcp" + + for harmless in (".", ".."): + naive = Path(os.path.normpath(root / f"harness.{harmless}.pointer")) + assert naive.parent == root + + assert Path(os.path.normpath(root / "harness.a/b.pointer")).parent != root + escaped = Path(os.path.normpath(root / "harness.../../../evil.pointer")) + assert not escaped.is_relative_to(root) + + +class TestSourcedComponent: + """The ``(source_name, component_id)`` pair, built where it belongs. + + ``tests/test_no_builtin_harness_source.py:69-75`` forbids the harness + source from ``components/`` and says why in its own failure message: + "Cross-source namespacing belongs to the resolution layer, keyed by a + (source_name, component_id) pair, and never enters ``ComponentSpec.id``." + This class is that sentence, executable. + """ + + def test_is_a_frozen_slots_dataclass_of_source_and_spec(self) -> None: + """Two fields, in that order, and no instance ``__dict__``.""" + assert dataclasses.is_dataclass(harness.SourcedComponent) + params = harness.SourcedComponent.__dataclass_params__ + assert params.frozen is True + assert "__slots__" in vars(harness.SourcedComponent) + names = tuple(f.name for f in dataclasses.fields(harness.SourcedComponent)) + assert names == ("source", "spec") + + def test_carries_the_component_id_unchanged(self) -> None: + """A component out of ``official`` still has id ``provider.demo``.""" + spec = _provider("demo", "demo.plane:DemoPlane") + sourced = harness.SourcedComponent(source="official", spec=spec) + assert sourced.source == "official" + assert sourced.spec is spec + assert sourced.spec.id == "provider.demo" + assert sourced.spec.name == "demo" + + def test_the_namespaced_id_is_not_even_constructible(self) -> None: + """Why the pair exists: ``ComponentSpec`` refuses the other design. + + Recorded here rather than assumed — the day ``models.py`` relaxes + this, the fold's whole shape is back on the table. + """ + with pytest.raises(CatalogError): + ComponentSpec( + kind=ComponentKind.PROVIDER, + name="demo", + id="official.provider.demo", + path="providers/demo/plane.py", + entrypoint="demo.plane:DemoPlane", + ) + + def test_assignment_to_either_field_raises(self) -> None: + """Frozen means the origin cannot drift away from its spec.""" + sourced = harness.SourcedComponent( + source="official", + spec=_provider("demo", "demo.plane:DemoPlane"), + ) + with pytest.raises(_ASSIGN_ERRORS): + sourced.source = "private" + with pytest.raises(_ASSIGN_ERRORS): + sourced.spec = _provider("other", "other.plane:OtherPlane") + + def test_no_attribute_holds_a_namespaced_id(self) -> None: + """The pair carries the origin beside the id, never folded into it.""" + sourced = harness.SourcedComponent( + source="official", + spec=_provider("demo", "demo.plane:DemoPlane"), + ) + assert not hasattr(sourced, "id") + + +class TestFoldComponents: + """First-wins on ``spec.id`` in source order; the loser is reported. + + For ``ComponentKind.PROVIDER`` an id collision *is* a plane-name + collision (``id == f"provider.{name}"``), so keying on the id is what + stops two ``WorkerProvider(name="demo")`` mounting under one namespace. + ``kept`` is the answer to that; there is deliberately no ``displaced`` + field — see :meth:`test_the_loser_is_reported_and_not_stored`. + """ + + def _two_sources( + self, + tmp_path: Path, + first: Sequence[ComponentSpec], + second: Sequence[ComponentSpec], + ) -> tuple[harness.Checkout, ...]: + return ( + _checkout(tmp_path, "official", _OFFICIAL_SHA, first), + _checkout(tmp_path, "private", _PRIVATE_SHA, second), + ) + + def test_the_first_source_wins_a_contested_id(self, tmp_path: Path) -> None: + """Two ``provider.demo`` rows, one kept, and it is the first file's.""" + winner = _provider("demo", "official.plane:DemoPlane") + loser = _provider("demo", "private.plane:DemoPlane") + checkouts = self._two_sources(tmp_path, [winner], [loser]) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert len(fold.kept) == 1 + kept = fold.kept[0] + assert kept.source == "official" + assert kept.spec.id == "provider.demo" + assert kept.spec.entrypoint == "official.plane:DemoPlane" + + def test_distinct_ids_are_kept_in_source_then_catalog_order( + self, tmp_path: Path + ) -> None: + """Source order outside, catalog order within — both, and only both.""" + alpha = _provider("alpha", "official.plane:Alpha") + beta = _provider("beta", "official.plane:Beta") + gamma = _provider("gamma", "private.plane:Gamma") + checkouts = self._two_sources(tmp_path, [alpha, beta], [gamma]) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert [(sc.source, sc.spec.id) for sc in fold.kept] == [ + ("official", "provider.alpha"), + ("official", "provider.beta"), + ("private", "provider.gamma"), + ] + + def test_only_the_requested_kind_is_folded(self, tmp_path: Path) -> None: + """A catalog is an inventory of every kind; one fold reads one kind.""" + checkouts = self._two_sources( + tmp_path, + [_skill("daily"), _provider("alpha", "official.plane:Alpha")], + [_skill("nightly")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert [sc.spec.id for sc in fold.kept] == ["provider.alpha"] + + def test_names_is_the_kept_component_name_set(self, tmp_path: Path) -> None: + """``fold.names`` is the set ``create_stack`` XORs entry points against.""" + checkouts = self._two_sources( + tmp_path, + [_provider("alpha", "official.plane:Alpha")], + [_provider("gamma", "private.plane:Gamma")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.names == frozenset({"alpha", "gamma"}) + + def test_a_contested_name_appears_once_in_names(self, tmp_path: Path) -> None: + """One mount per plane id, which is what a set of kept names buys.""" + checkouts = self._two_sources( + tmp_path, + [_provider("demo", "official.plane:DemoPlane")], + [_provider("demo", "private.plane:DemoPlane")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.names == frozenset({"demo"}) + assert len(fold.kept) == 1 + + def test_specs_from_returns_one_sources_specs_in_catalog_order( + self, tmp_path: Path + ) -> None: + """The overlay arm needs per-checkout grouping; this is that grouping.""" + alpha = _provider("alpha", "official.plane:Alpha") + beta = _provider("beta", "official.plane:Beta") + gamma = _provider("gamma", "private.plane:Gamma") + checkouts = self._two_sources(tmp_path, [alpha, beta], [gamma]) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.specs_from("official") == (alpha, beta) + assert fold.specs_from("private") == (gamma,) + + def test_specs_from_omits_a_displaced_spec(self, tmp_path: Path) -> None: + """The loser is not kept, so its own source does not report it either.""" + checkouts = self._two_sources( + tmp_path, + [_provider("demo", "official.plane:DemoPlane")], + [_provider("demo", "private.plane:DemoPlane")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.specs_from("private") == () + + def test_specs_from_an_unknown_source_is_empty(self, tmp_path: Path) -> None: + """A source nobody folded has no specs — an empty tuple, not a raise.""" + checkouts = self._two_sources( + tmp_path, + [_provider("alpha", "official.plane:Alpha")], + [_provider("gamma", "private.plane:Gamma")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.specs_from("nobody") == () + + def test_the_fold_carries_the_checkouts_it_was_folded_from( + self, tmp_path: Path + ) -> None: + """One owner of ``source -> tree``: the ``Checkout`` objects themselves. + + ``checkout_planes(fold)`` takes one argument because of this. A + parallel map would be a second owner of a fact ``Checkout`` already + holds, and a fold built from a different list would answer ``()`` from + ``specs_from`` with no error at all. + """ + checkouts = self._two_sources( + tmp_path, + [_provider("alpha", "official.plane:Alpha")], + [_provider("gamma", "private.plane:Gamma")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.checkouts == tuple(checkouts) + assert [checkout.source for checkout in fold.checkouts] == [ + "official", + "private", + ] + + def test_no_checkouts_folds_to_an_empty_result(self) -> None: + """No harness source is not an error; it is the empty fold.""" + fold = harness.fold_components((), ComponentKind.PROVIDER) + + assert fold.checkouts == () + assert fold.kept == () + assert fold.names == frozenset() + assert fold.specs_from("official") == () + + def test_the_loser_is_reported_and_not_stored( + self, tmp_path: Path, caplog: pytest.LogCaptureFixture + ) -> None: + """Exactly one warning, naming winner, loser and the contested id. + + There is no ``displaced`` field: nothing in production would read it, + and this repo's own first-wins precedents + (``discovery/overlay/catalog.py:83``, ``conventions.py:95``) drop + losers without recording them. The warning is what earns its keep; a + field whose only reader is a test does not. + """ + checkouts = self._two_sources( + tmp_path, + [_provider("demo", "official.plane:DemoPlane")], + [_provider("demo", "private.plane:DemoPlane")], + ) + + with caplog.at_level(logging.WARNING, logger=_LOGGER): + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + records = _warnings(caplog) + assert len(records) == 1 + message = records[0].getMessage() + assert "official" in message + assert "private" in message + assert "provider.demo" in message + assert not hasattr(fold, "displaced") + assert tuple(f.name for f in dataclasses.fields(fold)) == ( + "checkouts", + "component_roots", + "kept", + ) + + def test_an_uncontested_fold_reports_nothing( + self, tmp_path: Path, caplog: pytest.LogCaptureFixture + ) -> None: + """A warning per ordinary serve would train the operator to ignore it.""" + checkouts = self._two_sources( + tmp_path, + [_provider("alpha", "official.plane:Alpha")], + [_provider("gamma", "private.plane:Gamma")], + ) + + with caplog.at_level(logging.WARNING, logger=_LOGGER): + harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert _warnings(caplog) == [] + + def test_the_fold_is_frozen_and_slotted(self, tmp_path: Path) -> None: + """``names`` is derived, and neither stored field can be rebound.""" + checkouts = self._two_sources( + tmp_path, + [_provider("alpha", "official.plane:Alpha")], + [_provider("gamma", "private.plane:Gamma")], + ) + + fold = harness.fold_components(checkouts, ComponentKind.PROVIDER) + + assert fold.__dataclass_params__.frozen is True + assert "__slots__" in vars(harness.ComponentFold) + assert isinstance(harness.ComponentFold.names, property) + with pytest.raises(_ASSIGN_ERRORS): + fold.kept = () + with pytest.raises(_ASSIGN_ERRORS): + fold.checkouts = () + + +class TestComponentFold: + """One base per source, stored as the authored string and joined once. + + ``fold_components`` is the subject of the class above; this one is + :class:`~molmcp.harness.ComponentFold` itself, because four of the five + disagreeing constructions its ``__post_init__`` refuses cannot be reached + through the folder at all and have to be built by hand. + + **Why the raw string is stored and not the join.** ``tree`` already lives + on the ``Checkout`` objects the fold carries, so a stored + ``tree / component_root`` would be a second copy of a fact the object + already holds — the parallel ``source -> tree`` map ``ComponentFold``'s + own docstring argues against — and the invariant would then exist only to + police the agreement between two copies of one fact. With the string + stored and the join performed inside ``root_for`` against *that source's + own* ``Checkout.tree``, "the base belongs to the right tree" is a + **theorem** rather than an assertion: there is no other tree ``root_for`` + can reach. That is why nothing below tests a base pointing at an + unrelated path — the state is not representable, so there is nothing to + assert about it. + """ + + def _two_sources( + self, + tmp_path: Path, + *, + official_root: str = "", + private_root: str = "", + ) -> tuple[harness.Checkout, harness.Checkout]: + """``official`` then ``private``, one provider row each, real files.""" + return ( + _checkout( + tmp_path, + "official", + _OFFICIAL_SHA, + [_provider("alpha", "official.plane:Alpha")], + component_root=official_root, + ), + _checkout( + tmp_path, + "private", + _PRIVATE_SHA, + [_provider("gamma", "private.plane:Gamma")], + component_root=private_root, + ), + ) + + def test_a_rooted_catalog_answers_the_tree_joined_to_its_component_root( + self, tmp_path: Path + ) -> None: + """``component_root = "plugins/mol"`` answers ``tree/plugins/mol``. + + The expected path is spelled segment by segment rather than as the + input string re-joined, so the assertion is not the implementation + written twice. + """ + official, _ = self._two_sources(tmp_path, official_root="plugins/mol") + + fold = harness.fold_components((official,), ComponentKind.PROVIDER) + + assert fold.root_for("official") == official.tree / "plugins" / "mol" + + def test_a_rootless_catalog_answers_exactly_the_tree(self, tmp_path: Path) -> None: + """No key means the tree object itself, not another spelling of it. + + Path equality against the tree *this test built* is the assertion, + so anything that is not that exact :class:`~pathlib.Path` fails — + including a string carrying a stray ``.`` component or a trailing + separator. This is what keeps every install that has no + ``component_root`` today resolving byte-identical paths tomorrow. + """ + official, _ = self._two_sources(tmp_path) + + fold = harness.fold_components((official,), ComponentKind.PROVIDER) + + base = fold.root_for("official") + assert base == official.tree + assert base.is_dir() + + def test_each_source_is_answered_with_its_own_base(self, tmp_path: Path) -> None: + """One rooted source and one rootless source, in one fold. + + This is the case a *global* application of ``component_root`` gets + wrong. Applied to the fold rather than per source, the rootless + source's components would resolve under a directory its own catalog + never named — and its neighbour's would resolve correctly, which is + exactly the half-working install that is hardest to diagnose. + """ + official, private = self._two_sources(tmp_path, official_root="plugins/mol") + + fold = harness.fold_components((official, private), ComponentKind.PROVIDER) + + assert fold.root_for("official") == official.tree / "plugins" / "mol" + assert fold.root_for("private") == private.tree + + def test_component_roots_holds_the_authored_string_not_the_join( + self, tmp_path: Path + ) -> None: + """The field is ``source -> str``, in source order, verbatim. + + A joined ``Path`` here would be the parallel map the type refuses to + carry; the string is the fold's own datum, because no ``Checkout`` + holds it — ``activated_checkouts`` reads no catalog. + """ + official, private = self._two_sources(tmp_path, official_root="plugins/mol") + + fold = harness.fold_components((official, private), ComponentKind.PROVIDER) + + assert fold.component_roots == ( + ("official", "plugins/mol"), + ("private", ""), + ) + + def test_component_roots_has_no_default(self) -> None: + """A fold cannot be built without saying what each source's base is. + + A default would make the field's absence mean "every source is + rootless", which is a wrong answer rather than a missing one. + """ + fields = {f.name: f for f in dataclasses.fields(harness.ComponentFold)} + assert "component_roots" in fields + assert fields["component_roots"].default is dataclasses.MISSING + assert fields["component_roots"].default_factory is dataclasses.MISSING + with pytest.raises(TypeError): + harness.ComponentFold(checkouts=(), kept=()) + + def test_root_for_an_unknown_source_is_refused(self, tmp_path: Path) -> None: + """``unknown-source: 'nobody'`` — the register ``get`` already uses. + + ``specs_from`` tolerates an unknown source and answers ``()``; + ``root_for`` deliberately does not copy that tolerance. There is no + empty ``Path`` a caller could use, and a wrong base is the + half-applied failure this whole design exists to prevent. + """ + official, private = self._two_sources(tmp_path) + + fold = harness.fold_components((official, private), ComponentKind.PROVIDER) + + with pytest.raises(CatalogError) as excinfo: + fold.root_for("nobody") + + message = str(excinfo.value) + assert "unknown-source" in message + assert repr("nobody") in message + + def test_a_source_missing_from_component_roots_is_refused( + self, tmp_path: Path + ) -> None: + """Every checkout must have a base; a fold cannot answer for two.""" + official, private = self._two_sources(tmp_path) + + with pytest.raises(CatalogError): + harness.ComponentFold( + checkouts=(official, private), + component_roots=(("official", ""),), + kept=(), + ) + + def test_a_misnamed_source_in_component_roots_is_refused( + self, tmp_path: Path + ) -> None: + """A typo names a source nothing folded, and leaves one unanswered.""" + official, private = self._two_sources(tmp_path) + + with pytest.raises(CatalogError): + harness.ComponentFold( + checkouts=(official, private), + component_roots=(("official", ""), ("privte", "plugins/mol")), + kept=(), + ) + + def test_an_extra_source_in_component_roots_is_refused( + self, tmp_path: Path + ) -> None: + """A base for a source this fold was not built from answers nobody.""" + official, _ = self._two_sources(tmp_path) + + with pytest.raises(CatalogError): + harness.ComponentFold( + checkouts=(official,), + component_roots=(("official", ""), ("private", "plugins/mol")), + kept=(), + ) + + def test_a_duplicated_source_in_component_roots_is_refused( + self, tmp_path: Path + ) -> None: + """The clause this one needs is uniqueness on the *roots* side. + + Set equality alone admits it — ``{"official", "private"}`` on both + sides — and ``root_for``'s linear scan would then answer with + whichever entry it met first, silently, while a second entry naming + the same source said something else. + """ + official, private = self._two_sources(tmp_path) + + with pytest.raises(CatalogError): + harness.ComponentFold( + checkouts=(official, private), + component_roots=( + ("official", ""), + ("official", "plugins/mol"), + ("private", ""), + ), + kept=(), + ) + + def test_two_checkouts_sharing_a_source_name_are_refused( + self, tmp_path: Path + ) -> None: + """The clause this one needs is uniqueness on the *checkouts* side. + + Set equality holds and ``component_roots`` is unique, so every + narrower invariant admits this pair — and ``root_for``'s scan would + answer with the first checkout's tree while the second one's + components resolved nowhere. ``activated_checkouts`` already refuses + a duplicate source name, but ``ComponentFold`` is directly + constructible and cannot rely on its own caller. + """ + first = _checkout( + tmp_path / "first", + "official", + _OFFICIAL_SHA, + [_provider("alpha", "official.plane:Alpha")], + ) + second = _checkout( + tmp_path / "second", + "official", + _PRIVATE_SHA, + [_provider("gamma", "other.plane:Gamma")], + ) + assert first.tree != second.tree + + with pytest.raises(CatalogError): + harness.ComponentFold( + checkouts=(first, second), + component_roots=(("official", ""),), + kept=(), + ) + + +class TestCheckoutPlanes: + """The provider arm, over a real tree: half a harness made unreachable. + + A fold has two consumers — this one and the overlay seam in + ``molmcp.runtime`` — and the failure ``component_root`` exists to kill is + being applied in one of them and forgotten in the other. Providers that + resolve while overlays do not is far harder to diagnose than an install + that resolves nothing, because it looks like it works. This class is the + provider half; ``tests/test_stack.py`` owns the overlay half, which is + ``create_stack``'s composition rather than this module's contract. + """ + + def test_a_rooted_provider_is_imported_from_under_the_component_root( + self, tmp_path: Path + ) -> None: + """``plugins/mol`` + ``providers/demo/plane.py`` — one directory. + + The module file is really planted, so ``_import_root`` takes its + a-file-hands-back-its-parent branch rather than the directory branch, + and the resolved base is the one a child process would import from. + """ + checkout = _checkout( + tmp_path, + "official", + _OFFICIAL_SHA, + [_provider("demo", "demo.plane:DemoPlane")], + component_root="plugins/mol", + ) + module = checkout.tree / "plugins" / "mol" / "providers" / "demo" / "plane.py" + module.parent.mkdir(parents=True) + module.write_text("", encoding="utf-8") + + planes = harness.checkout_planes( + harness.fold_components((checkout,), ComponentKind.PROVIDER) + ) + + assert len(planes) == 1 + assert planes[0].name == "demo" + # ``WorkerProvider`` keeps its import root private and publishes only + # ``probe()``, so the exact answer is read off ``_path`` and the + # public consequence is asserted beside it: a base under the wrong + # directory is a directory that does not exist, which is what an + # operator actually meets when the two arms disagree. + assert Path(planes[0]._path) == module.parent + assert planes[0].probe() is True + + +class TestActivatedCheckouts: + """The real function, over a real store and real pointer files. + + Nothing in this class monkeypatches ``Activation``, ``ImmutableGitStore``, + ``GitHubTransport``, ``load_harness_catalog`` or ``WorkerProvider`` — the + five names ``tests/test_stack.py``'s ``_wire`` seam replaces. A suite built + only on that seam proves the composition *order* and nothing whatsoever + about those five, which is not hypothetical: link 01 left ``molmcp serve`` + broken for every install while 1852 tests passed, because the only + occurrence of ``_harness_locator`` under ``tests/`` was a test *name* + (``notes.md:faked-seam-hides-broken-reader``). + + So the store is planted by hand — ``/harness/commits//`` with a + ``metadata.json`` and a ``tree/`` — and the pointers are literal version-1 + JSON. Nothing fetches: ``GitHubTransport.__init__`` (``git.py:82-89``) + stores a token and does no I/O, and ``publish`` is never called. + + The planted trees are left **empty**, deliberately. ``activated_checkouts`` + binds a pointer and hands back a tree; reading ``harness.toml`` is + ``fold_components``' job. A tree with no catalog in it is how an + implementation that read one here would be caught. + """ + + def test_the_hand_written_pointer_is_the_records_own_shape(self) -> None: + """The plant is checked against the contract, not against a memory. + + Every pointer in this class is written as a JSON literal, so the two + facts the per-source-pointer route was chosen to preserve — version 1, + and exactly these four keys — have to be asserted somewhere or the + whole class could drift away from ``activate.py`` while staying green. + """ + payload = _pointer_payload(_OFFICIAL_SHA) + assert set(payload) == _POINTER_KEYS + assert payload["version"] == 1 + + def test_two_sources_yield_two_checkouts_in_file_order( + self, tmp_path: Path + ) -> None: + """Each named source contributes its own commit, its own tree, its own name. + + This is the line link 01 lost: ``server.py:336`` consumed the ordered + tuple of sources as a *boolean* and then bound one pointer, so a second + entry changed nothing about what was served. + """ + config, root = _config_and_root(tmp_path) + official_tree = _publish_by_hand(root / "harness", _OFFICIAL_SHA) + private_tree = _publish_by_hand( + root / "harness", _PRIVATE_SHA, owner="acme", repo="tooling" + ) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "private"), _PRIVATE_SHA) + + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert isinstance(checkouts, tuple) + assert [checkout.source for checkout in checkouts] == ["official", "private"] + assert [checkout.sha for checkout in checkouts] == [ + _OFFICIAL_SHA, + _PRIVATE_SHA, + ] + assert [checkout.tree for checkout in checkouts] == [ + official_tree, + private_tree, + ] + assert all(checkout.tree.is_dir() for checkout in checkouts) + + def test_the_order_is_the_settings_list_order(self, tmp_path: Path) -> None: + """File order, not directory order: the operator's priority control. + + ``fold_components`` resolves a contested id first-wins over this + sequence, so the order this function returns is the only thing + deciding which source's ``provider.demo`` gets served. + """ + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _publish_by_hand(root / "harness", _PRIVATE_SHA, owner="acme", repo="tooling") + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "private"), _PRIVATE_SHA) + + checkouts = harness.activated_checkouts( + config, + (_source("private", locator="acme/tooling"), _source("official")), + ) + + assert [checkout.source for checkout in checkouts] == ["private", "official"] + assert [checkout.sha for checkout in checkouts] == [ + _PRIVATE_SHA, + _OFFICIAL_SHA, + ] + + def test_exactly_one_commits_directory_holds_every_activated_sha( + self, tmp_path: Path + ) -> None: + """One store, several pointers — the whole shape of this link. + + ``ImmutableGitStore`` records provenance per SHA and refuses a SHA + claimed by a second repository, so a per-source root would buy nothing + and would strand every already-published tree. The pointers are what + multiply, and they are plain siblings of the one store root. + """ + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _publish_by_hand(root / "harness", _PRIVATE_SHA, owner="acme", repo="tooling") + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "private"), _PRIVATE_SHA) + + harness.activated_checkouts(config, (_source("official"), _source("private"))) + + commits = sorted(path for path in root.rglob("commits") if path.is_dir()) + assert commits == [root / "harness" / "commits"] + assert sorted(path.name for path in commits[0].iterdir()) == sorted( + [_OFFICIAL_SHA, _PRIVATE_SHA] + ) + assert sorted(path.name for path in root.iterdir()) == [ + "harness", + "harness.official.pointer", + "harness.private.pointer", + ] + + def test_two_sources_activating_one_sha_share_the_one_tree( + self, tmp_path: Path + ) -> None: + """Two pointers may name the same commit; the store still holds it once.""" + config, root = _config_and_root(tmp_path) + tree = _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "private"), _OFFICIAL_SHA) + + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert [checkout.source for checkout in checkouts] == ["official", "private"] + assert {checkout.tree for checkout in checkouts} == {tree} + assert [path.name for path in (root / "harness" / "commits").iterdir()] == [ + _OFFICIAL_SHA + ] + + @pytest.mark.parametrize( + ("activated", "absent"), + [ + pytest.param("official", "private", id="second-source-unactivated"), + pytest.param("private", "official", id="first-source-unactivated"), + ], + ) + def test_a_source_with_no_pointer_file_is_skipped( + self, tmp_path: Path, activated: str, absent: str + ) -> None: + """A named-but-unactivated source is not an error; it contributes nothing. + + Nothing activated serves exactly like an unset locator, and it does so + *per source*: the neighbour still yields its checkout. The missing + pointer is also not created on the way past — ``Activation.bind`` turns + a missing file into an in-memory empty record and writes nothing, and + serving must never be the thing that writes an activation. + """ + config, root = _config_and_root(tmp_path) + shas = {"official": _OFFICIAL_SHA, "private": _PRIVATE_SHA} + _publish_by_hand(root / "harness", shas[activated]) + _write_pointer(harness.pointer_path(root, activated), shas[activated]) + + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert [checkout.source for checkout in checkouts] == [activated] + assert checkouts[0].sha == shas[activated] + assert not harness.pointer_path(root, absent).exists() + + def test_a_pointer_with_no_active_sha_contributes_nothing( + self, tmp_path: Path + ) -> None: + """A pointer file that exists but activates nothing is the same skip.""" + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "private"), None) + + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert [checkout.source for checkout in checkouts] == ["official"] + + def test_no_sources_is_the_empty_result_and_touches_no_disk( + self, tmp_path: Path + ) -> None: + """The un-harnessed install: no checkout, and no cache root created.""" + config, root = _config_and_root(tmp_path) + + assert harness.activated_checkouts(config, ()) == () + assert not root.exists() + + def test_an_unpublished_sha_names_both_the_sha_and_its_source( + self, tmp_path: Path + ) -> None: + """The error identifies *which* source is broken, not just the SHA. + + Naming the SHA and the store root identifies nothing under N sources: + the operator has to know which entry of the ``harness`` list to go and + fix. A missing tree is named rather than silently re-fetched, because + serving a different commit than the one that was activated is the one + outcome nobody asked for. + + The second source is named ``acme`` rather than ``private`` on purpose: + on darwin ``tmp_path`` lives under ``/private/var``, so ``"private" in + message`` would pass on the store root alone and prove nothing. + """ + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "acme"), _PRIVATE_SHA) + + with pytest.raises(ConfigurationError) as excinfo: + harness.activated_checkouts( + config, + (_source("official"), _source("acme", locator="acme/tooling")), + ) + + message = str(excinfo.value) + assert _PRIVATE_SHA in message + assert "acme" in message + # The healthy neighbour is not implicated in its neighbour's failure. + assert _OFFICIAL_SHA not in message + + def test_two_entries_sharing_a_name_are_refused(self, tmp_path: Path) -> None: + """One name, one pointer file: two entries under it is not resolvable. + + Two sources named alike would share ``harness..pointer`` — so the + second silently serves whatever the first activated — and would make + ``ComponentFold.specs_from(source)`` ambiguous. ``collection/index.py:75`` + is the precedent: a duplicate *origin* name is the one hard error. + """ + config, root = _config_and_root(tmp_path) + before = _entries(tmp_path) + + with pytest.raises(ConfigurationError) as excinfo: + harness.activated_checkouts( + config, + (_source("official"), _source("official", locator="acme/tool")), + ) + + assert "official" in str(excinfo.value) + assert _entries(tmp_path) == before + assert not root.exists() + + def test_two_names_differing_only_in_case_are_refused(self, tmp_path: Path) -> None: + """``official`` and ``Official`` are one pointer file on this platform. + + The comparison is ``casefold()``, not equality: ``HarnessSource`` + deliberately permits ``MolCrafts`` casing, so an exact check passes + this pair — and on darwin (this repo's dev platform) and on Windows + both names map to one file, which is exactly the hazard the check + exists for. + + Both spellings must appear in the message. Naming only the casefolded + key points at neither line of the settings file the operator has to + edit, and the whole value of this error is sending them there. + """ + config, _ = _config_and_root(tmp_path) + + with pytest.raises(ConfigurationError) as excinfo: + harness.activated_checkouts( + config, + (_source("official"), _source("Official", locator="acme/tool")), + ) + + message = str(excinfo.value) + assert "official" in message + assert "Official" in message + + def test_an_unusable_source_name_is_refused_before_anything_is_read( + self, tmp_path: Path + ) -> None: + """The traversal guard is reached through this function, not only directly. + + ``HarnessSource(name="a/b")`` constructs today — ``settings.py:152-159`` + excludes ``name`` from the ``/`` rejection — so ``pointer_path``'s guard + is the only thing between a settings file and a write outside the cache + root. A ``TestPointerPath`` that passed while this function built its + paths by hand would prove nothing. + """ + config, root = _config_and_root(tmp_path) + root.mkdir(parents=True) + before = _entries(tmp_path) + + with pytest.raises(ConfigurationError) as excinfo: + harness.activated_checkouts(config, (_source("a/b"),)) + + assert repr("a/b") in str(excinfo.value) + assert _entries(tmp_path) == before + + def test_a_stale_legacy_pointer_is_named_once_and_never_read( + self, tmp_path: Path, caplog: pytest.LogCaptureFixture + ) -> None: + """``/harness.pointer`` is reported, and contributes no checkout. + + The legacy file here names a SHA that *is* published, so an + implementation that fell back to reading it would hand back a checkout + and fail this test on the empty result rather than on the warning. That + is the assertion that matters: authority stays unambiguous because the + legacy file is never read, the shape ``CLAUDE.md``'s stranded-orphan + rule asks for. + + The probe is one ``legacy.exists()`` plus one ``pointer_path(...)`` + existence check per source, evaluated once *before* the per-source + loop — filesystem contact this function otherwise never makes. + """ + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(root / "harness.pointer", _OFFICIAL_SHA) + + with caplog.at_level(logging.WARNING, logger=_LOGGER): + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert checkouts == () + records = _warnings(caplog) + assert len(records) == 1 + message = records[0].getMessage() + assert "harness.pointer" in message + + def test_a_half_migrated_install_is_not_warned( + self, tmp_path: Path, caplog: pytest.LogCaptureFixture + ) -> None: + """Stale file plus one activated source: no warning, deliberately. + + Nothing in the product ever writes ``harness.pointer`` — there is no + caller of ``Activation.stage`` / ``promote`` / ``rollback`` anywhere in + ``src/`` — so one notice at the point it can still matter is the whole + budget. A test that did not pin this would let someone "helpfully" make + the probe unconditional, and a warning on every ordinary serve trains + the operator to ignore the one that mattered. + """ + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(root / "harness.pointer", _PRIVATE_SHA) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + + with caplog.at_level(logging.WARNING, logger=_LOGGER): + checkouts = harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert [checkout.source for checkout in checkouts] == ["official"] + assert [checkout.sha for checkout in checkouts] == [_OFFICIAL_SHA] + assert _warnings(caplog) == [] + + def test_an_install_with_no_legacy_pointer_reports_nothing( + self, tmp_path: Path, caplog: pytest.LogCaptureFixture + ) -> None: + """The ordinary serve is silent — including the unactivated source.""" + config, root = _config_and_root(tmp_path) + _publish_by_hand(root / "harness", _OFFICIAL_SHA) + _write_pointer(harness.pointer_path(root, "official"), _OFFICIAL_SHA) + + with caplog.at_level(logging.WARNING, logger=_LOGGER): + harness.activated_checkouts( + config, (_source("official"), _source("private")) + ) + + assert _warnings(caplog) == [] + + +#: ``user.name`` / ``user.email`` for the one commit ``_git_checkout`` makes. +#: Passed per invocation rather than configured, so no developer's global git +#: identity is read and none is written into ``tmp_path``. +_GIT_IDENTITY = ( + "-c", + "user.name=molmcp tests", + "-c", + "user.email=tests@molmcp.invalid", +) + + +def _run_git(root: Path, *args: str) -> None: + subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + text=True, + ) + + +def _git_checkout(root: Path) -> Path: + """Create *root* as a real one-commit git repository and return it. + + ``assert_servable`` probes for ``.git`` under the root it is handed, so a + directory holding an empty file of that name would satisfy the letter of + the check. A real repository is planted anyway, because the refusals below + turn on it: each of them puts a checkout **that really works** at the + location a working-directory-relative spelling names, so the claim under + test is "refused even though it resolves to something usable from where + this process happens to stand" rather than "refused because nothing is + there". + + Mirrored from ``tests/test_stack.py``'s helper of the same name rather + than imported from it: that is a private name in a module mirroring a + different production unit, and the three lines are cheaper than coupling + two suites together. + """ + root.mkdir(parents=True, exist_ok=True) + _run_git(root, "init", "-q", "--initial-branch=main") + (root / "harness.toml").write_text("", encoding="utf-8") + _run_git(root, "add", "-A") + _run_git(root, *_GIT_IDENTITY, "commit", "--no-gpg-sign", "-q", "-m", "first") + return root + + +def _hermetic_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point ``~`` at an empty temporary tree and return it. + + Both spellings of "the user's home" are aimed at the same directory: + :meth:`Path.home`, which is how the rest of this package finds it, and the + ``HOME`` / ``USERPROFILE`` environment that :func:`os.path.expanduser` + consults — ``Path.expanduser`` delegates to that function and does **not** + go through ``Path.home``. Pinning both keeps these tests on the behaviour + (a ``~`` path names one directory in every session) instead of on which of + the two APIs the expansion happens to be written with. + """ + home = tmp_path / "home" + home.mkdir(parents=True, exist_ok=True) + monkeypatch.setattr(Path, "home", staticmethod(lambda: home)) + monkeypatch.setenv("HOME", str(home)) + monkeypatch.setenv("USERPROFILE", str(home)) + return home + + +def _working_directory(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Stand the process in a temporary project directory and return it. + + This is the directory an MCP client's ``molmcp serve`` would inherit — + one of many, differing per session, and the thing a ``path`` entry may + not be read against. + """ + project = tmp_path / "project" + project.mkdir(parents=True, exist_ok=True) + monkeypatch.chdir(project) + return project + + +class TestAssertServable: + """A parsed locator is servable, or it is not constructible. + + This function is the single owner of the servability rule — ``molmcp + serve`` reaches it through ``server._harness_locator`` and ``molmcp + harness sync`` calls it on the one entry it was named. GitHub locators + are complete without a path. Local locators must name a checkout. + Relative spellings never arrive here: ``HarnessSource`` refuses them at + construction as ``LocatorError``. + """ + + def test_harness_coordinates_is_gone_from_the_module(self) -> None: + assert not hasattr(harness, "HARNESS_COORDINATES") + + def test_a_github_locator_is_servable_without_a_path(self) -> None: + harness.assert_servable( + HarnessSource(name="official", locator="MolCrafts/harness@main") + ) + + def test_an_absolute_checkout_is_servable(self, tmp_path: Path) -> None: + """The unambiguous local spelling: one directory, no context.""" + checkout = _git_checkout(tmp_path / "checkout") + + harness.assert_servable(HarnessSource(name="mine", locator=str(checkout))) + + def test_an_absolute_path_that_is_no_checkout_is_still_refused( + self, tmp_path: Path + ) -> None: + with pytest.raises(ConfigurationError) as excinfo: + harness.assert_servable( + HarnessSource(name="mine", locator=str(tmp_path / "gone")) + ) + + assert "mine" in str(excinfo.value) + + @pytest.mark.parametrize("spelling", ["./checkout", "../harness"]) + def test_a_relative_locator_is_a_parse_time_error(self, spelling: str) -> None: + """Relative paths never become cwd-relative servable sources.""" + with pytest.raises((LocatorError, ValueError)): + HarnessSource(name="mine", locator=spelling) + + def test_enable_empty_tuple_is_still_servable(self) -> None: + """Slice 01 stores ``enable`` and does not filter on it.""" + harness.assert_servable( + HarnessSource( + name="official", + locator="MolCrafts/harness@main", + enable=(), + ) + ) + + def test_a_home_relative_path_naming_a_checkout_is_servable( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """``~/harness`` is a local locator and names one directory.""" + home = _hermetic_home(tmp_path, monkeypatch) + _working_directory(tmp_path, monkeypatch) + _git_checkout(home / "harness") + + harness.assert_servable(HarnessSource(name="mine", locator="~/harness")) + + def test_a_home_relative_path_is_refused_when_home_holds_no_checkout( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """``~`` expands to home and nowhere else.""" + _hermetic_home(tmp_path, monkeypatch) + project = _working_directory(tmp_path, monkeypatch) + _git_checkout(project / "harness") + + with pytest.raises(ConfigurationError) as excinfo: + harness.assert_servable(HarnessSource(name="mine", locator="~/harness")) + + message = str(excinfo.value) + assert "mine" in message + + def test_the_stored_locator_is_not_rewritten_by_the_check( + self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + """The operator's locator string is unchanged by the probe.""" + home = _hermetic_home(tmp_path, monkeypatch) + _working_directory(tmp_path, monkeypatch) + _git_checkout(home / "harness") + source = HarnessSource(name="mine", locator="~/harness") + + harness.assert_servable(source) + + assert source.locator == "~/harness" diff --git a/tests/test_harness_agents.py b/tests/test_harness_agents.py new file mode 100644 index 0000000..23009a8 --- /dev/null +++ b/tests/test_harness_agents.py @@ -0,0 +1,264 @@ +"""The two harness agents are the evaluator's fixed instruments. + +An evaluation compares two harnesses, which only means something if +everything else holds still. Two things can move without anyone noticing. +A ``model:`` that resolves from the environment makes two runs a week apart +incomparable — the judge would have changed along with the defendant. And an +actor that can read the criteria optimises for them: the reading then measures +test-taking, not whether the harness leads a person to the right move on its +own. + +Neither file is code, so nothing else in this repo would ever complain about +them. These are structural assertions on the text, in the manner of +``test_no_env_switches.py``: read the file by path, parse the frontmatter with +a few lines of string handling rather than a YAML dependency the repo does not +have, and say plainly what is wrong. +""" + +from __future__ import annotations + +import importlib +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parents[1] +AGENTS = REPO / ".claude" / "agents" + +ACTOR = AGENTS / "harness-actor.md" +OBSERVER = AGENTS / "harness-observer.md" + +#: The four frontmatter keys a subagent definition pins (ac-012). +_REQUIRED_KEYS = ("name", "description", "tools", "model") + +#: A templated model id would let the subject drift with the environment. +_PLACEHOLDER = "{{" + +#: Tools that would let one evaluation round change the repository it runs in. +_ACTOR_MUST_NOT_HOLD = ("Write", "Edit") + +#: The vocabulary of the criteria. The actor is told the task, nothing else. +_CRITERIA_WORDS = ("expect", "forbid", "harness_cases") + +#: Every key of the observation schema the observer is the sole source of. +_OBSERVATION_KEYS = ( + "case_id", + "seed", + "side", + "contract_met", + "tool_errors", + "call_count", +) + +#: Naming a side is proof the blind was broken before the observer wrote. +_SIDE_NAMES = ("champion", "challenger") + +#: Readings that cannot be taken off a transcript, and so must not be invited. +_UNOBSERVABLE = ("tokens", "latency_s") + + +def _rel(path: Path) -> str: + return path.relative_to(REPO).as_posix() + + +def _read(path: Path) -> str: + """The file's text, or a readable failure instead of an OSError traceback.""" + assert path.is_file(), ( + f"{_rel(path)} does not exist. The harness evaluator needs both agent " + f"definitions in the repository — an observer that lives in the tree " + f"under test would change along with it." + ) + return path.read_text(encoding="utf-8") + + +def _frontmatter(path: Path) -> dict[str, str]: + """The ``key: value`` lines between the opening and closing ``---`` fences. + + Enough YAML for four scalar keys plus a tool list written inline + (``Read, Grep``), bracketed (``[Read, Grep]``) or as a ``- Read`` block. + The repo carries no YAML parser and this spec adds no dependency. + """ + lines = _read(path).splitlines() + + assert [line.strip() for line in lines[:1]] == ["---"], ( + f"{_rel(path)} must open with a --- YAML frontmatter fence; its first " + f"line is {lines[:1]!r}." + ) + + closing = next( + (i for i, line in enumerate(lines[1:], 1) if line.strip() == "---"), None + ) + assert closing is not None, ( + f"{_rel(path)} opens a --- frontmatter fence that is never closed by a " + f"second --- line." + ) + + fields: dict[str, str] = {} + key: str | None = None + for line in lines[1:closing]: + stripped = line.strip() + if not stripped or stripped.startswith("#"): + continue + if stripped.startswith("- ") and key is not None: + item = stripped[2:].strip() + fields[key] = ", ".join(part for part in (fields[key], item) if part) + continue + name, sep, value = stripped.partition(":") + if not sep: + continue + key = name.strip() + fields[key] = value.strip() + return fields + + +def _tool_names(value: str) -> tuple[str, ...]: + """Tool names out of whichever of the three list spellings was used.""" + parts = (part.strip().strip("\"'") for part in value.strip("[]").split(",")) + return tuple(part for part in parts if part) + + +def _cases() -> list[dict[str, object]]: + """The case set, which is the only source of the strings the actor may not see.""" + try: + module = importlib.import_module("harness_cases") + except ImportError as exc: + pytest.fail( + f"cannot import `harness_cases` ({exc}). The criteria live only " + f"there, so without it this guard would have nothing to look for " + f"and would pass by vacuity. Expected scripts/harness_cases.py " + f'with "scripts" on [tool.pytest.ini_options] pythonpath.' + ) + + raw = getattr(module, "CASES", None) + assert isinstance(raw, list) and raw, ( + f"harness_cases.CASES must be a non-empty list of cases; got {raw!r}." + ) + + cases: list[dict[str, object]] = [] + for entry in raw: + assert isinstance(entry, dict), f"CASES entry is not a dict: {entry!r}" + cases.append(entry) + return cases + + +def _criteria_strings() -> tuple[str, ...]: + """Every ``expect`` and ``forbid`` string across the case set.""" + strings: list[str] = [] + for case in _cases(): + for key in ("expect", "forbid"): + values = case.get(key) + assert isinstance(values, list) and values, ( + f"case {case.get('id')!r} needs a non-empty {key!r} list " + f"(tests/test_harness_cases.py owns that rule; this guard only " + f"reads the strings)." + ) + for value in values: + assert isinstance(value, str), ( + f"case {case.get('id')!r} has a non-string {key!r} entry: {value!r}" + ) + strings.append(value) + return tuple(strings) + + +class TestHarnessAgents: + @pytest.mark.parametrize("path", (ACTOR, OBSERVER), ids=lambda p: p.stem) + def test_frontmatter_carries_the_four_keys(self, path: Path): + fields = _frontmatter(path) + missing = [key for key in _REQUIRED_KEYS if key not in fields] + + assert missing == [], ( + f"{_rel(path)} frontmatter is missing {missing}. A subagent " + f"definition pins {list(_REQUIRED_KEYS)}; anything left out is " + f"resolved by the host, which is exactly the drift this evaluator " + f"is measuring against." + ) + + @pytest.mark.parametrize( + ("path", "expected"), + ((ACTOR, "harness-actor"), (OBSERVER, "harness-observer")), + ids=("actor", "observer"), + ) + def test_each_definition_names_itself(self, path: Path, expected: str): + assert _frontmatter(path).get("name") == expected, ( + f"{_rel(path)} must declare `name: {expected}`. The orchestrator " + f"dispatches on that name, not on the filename." + ) + + @pytest.mark.parametrize("path", (ACTOR, OBSERVER), ids=lambda p: p.stem) + def test_model_is_a_pinned_literal(self, path: Path): + model = _frontmatter(path).get("model", "") + + assert model and _PLACEHOLDER not in model, ( + f"{_rel(path)} must pin `model` to a non-empty literal carrying no " + f"{_PLACEHOLDER} placeholder; got {model!r}. The judge must not " + f"drift with the environment, and neither must the subject — two " + f"runs a week apart have to stay comparable." + ) + + def test_the_actor_cannot_write_or_edit(self): + tools = _tool_names(_frontmatter(ACTOR).get("tools", "")) + held = [tool for tool in _ACTOR_MUST_NOT_HOLD if tool in tools] + + assert held == [], ( + f"{_rel(ACTOR)} grants {held}. One evaluation round must not modify " + f"the repository it runs in, and both sides run this single " + f"definition, so any tool it holds is under test alongside the " + f"harness. Declared tools: {list(tools)}." + ) + + def test_the_actor_never_repeats_a_criterion(self): + body = _read(ACTOR).casefold() + leaked = [text for text in _criteria_strings() if text.casefold() in body] + + assert leaked == [], ( + f"{_rel(ACTOR)} repeats {len(leaked)} criteria string(s) from the " + f"case set, first {leaked[:1]!r}. An actor that knows the criteria " + f"optimises for them, and the reading measures test-taking instead " + f"of whether the harness leads to the right move on its own." + ) + + def test_the_actor_never_names_the_criteria_vocabulary(self): + body = _read(ACTOR).casefold() + named = [word for word in _CRITERIA_WORDS if word in body] + + assert named == [], ( + f"{_rel(ACTOR)} names {named}. The actor receives the task text and " + f"the harness under test; the moment it can name where the criteria " + f"live it can go and read them." + ) + + @pytest.mark.parametrize("label", ("A", "B")) + def test_the_observer_names_the_blind_labels(self, label: str): + body = _read(OBSERVER) + forms = (f'"{label}"', f"'{label}'", f"`{label}`", f"{label}/", f"/{label}") + + assert any(form in body for form in forms), ( + f"{_rel(OBSERVER)} must name the blind label {label} as a label — " + f"quoted, fenced, or as the pair A/B. The two transcripts reach the " + f"observer under these labels and leave it under them; the manifest " + f"is what maps them back." + ) + + @pytest.mark.parametrize("key", _OBSERVATION_KEYS) + def test_the_observer_names_every_observation_key(self, key: str): + assert key in _read(OBSERVER), ( + f"{_rel(OBSERVER)} never names the observation key {key!r}. The " + f"observer is the sole source of this schema; a key it is not told " + f"to emit is a cell the report cannot fill." + ) + + @pytest.mark.parametrize("name", _SIDE_NAMES) + def test_the_observer_cannot_name_a_side(self, name: str): + assert name not in _read(OBSERVER).casefold(), ( + f"{_rel(OBSERVER)} contains {name!r}. An observer that can name a " + f"side has been told which is which; unblinding belongs to the " + f"manifest, which the observer never sees." + ) + + @pytest.mark.parametrize("reading", _UNOBSERVABLE) + def test_the_observer_cannot_name_an_unobservable_reading(self, reading: str): + assert reading not in _read(OBSERVER).casefold(), ( + f"{_rel(OBSERVER)} contains {reading!r}. A reading that cannot be " + f"taken off a transcript is one the observer would have to invent, " + f"and invented telemetry is worse than the zero the report records." + ) diff --git a/tests/test_harness_cases.py b/tests/test_harness_cases.py new file mode 100644 index 0000000..adfd5a3 --- /dev/null +++ b/tests/test_harness_cases.py @@ -0,0 +1,217 @@ +"""The harness case set is data, and the actor is never told the criteria. + +``scripts/harness_cases.py`` is the only place this repo says what "a better +harness" means, so it is held to two rules that a reviewer cannot enforce by +reading. + +The first is that it stays plain Python. A case set in YAML or JSON needs a +parser, a schema and a second place to look before anyone can tell what the +evaluator actually asserts; the same list written as a literal needs none of +them, which is why ``tests/discovery/golden_queries.py`` is shaped this way +too. + +The second is the one the whole design rests on: an actor that can read the +criteria optimises for the criteria, and the report then measures exam +technique rather than whether the harness leads a real user to the right +move. "The actor never sees them" is a wish until something fails when a +criterion string turns up inside the task text that is handed over verbatim. + +Structural guards read the module as text (the habit of +``tests/test_no_env_switches.py``); the behavioural ones import it flat -- +``scripts`` is on pytest's ``pythonpath``. +""" + +from __future__ import annotations + +import ast +import sys +from pathlib import Path + +import pytest +from _ast_checks import reads_environment +from harness_cases import CASES, case_by_id, graduated_ids, held_out_ids + +#: The module under test, read as source for the structural guards. +_SOURCE = Path(__file__).resolve().parents[1] / "scripts" / "harness_cases.py" + +#: Every key a case carries, and nothing besides. +_KEYS = frozenset({"id", "graduated", "task", "expect", "forbid"}) + +#: Importing any of these would mean the case set had become a file format. +_SERIALISATION_MODULES = frozenset( + { + "configparser", + "csv", + "json", + "pickle", + "plistlib", + "ruamel", + "toml", + "tomli", + "tomllib", + "xml", + "yaml", + } +) + + +def _imported_roots() -> frozenset[str]: + """Top-level module names imported by ``scripts/harness_cases.py``. + + Returns: + The first dotted segment of every ``import`` and ``from`` target. A + relative import contributes its leading dots instead, which no + standard-library check can accept -- ``scripts/`` is not a package. + """ + tree = ast.parse(_SOURCE.read_text(encoding="utf-8")) + roots: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + roots.update(alias.name.split(".")[0] for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + roots.add("." * node.level) + elif node.module: + roots.add(node.module.split(".")[0]) + return frozenset(roots) + + +class TestHarnessCases: + def test_every_case_carries_exactly_the_five_keys(self): + wrong = [ + (index, sorted(set(case) ^ _KEYS)) + for index, case in enumerate(CASES) + if set(case) != _KEYS + ] + + assert wrong == [], ( + f"each case is exactly {sorted(_KEYS)}; an extra key is a field " + f"nothing reads and a missing one is a case the evaluator cannot " + f"place, offenders (index, symmetric difference): {wrong}" + ) + + def test_every_field_has_the_declared_type(self): + wrong: list[tuple[object, str]] = [] + for case in CASES: + case_id = case.get("id") + if not isinstance(case_id, str): + wrong.append((case_id, "id must be a str")) + if not isinstance(case.get("graduated"), bool): + wrong.append((case_id, "graduated must be a bool")) + if not isinstance(case.get("task"), str): + wrong.append((case_id, "task must be a str")) + for key in ("expect", "forbid"): + value = case.get(key) + if not isinstance(value, list) or not all( + isinstance(item, str) for item in value + ): + wrong.append((case_id, f"{key} must be a list[str]")) + + assert wrong == [], f"case fields have declared types: {wrong}" + + def test_ids_are_unique(self): + ids = [case["id"] for case in CASES] + + assert len(set(ids)) == len(ids), ( + f"case_by_id can only return one of two cases sharing an id, and " + f"the loser is silently never run: {ids}" + ) + + def test_expect_and_forbid_are_both_non_empty(self): + empty = [ + (case["id"], key) + for case in CASES + for key in ("expect", "forbid") + if not case[key] + ] + + assert empty == [], ( + f"a case with nothing to expect proves nothing and a case with no " + f"negative control can never fail, so it is not a case: {empty}" + ) + + def test_both_a_graduated_and_a_held_out_case_exist(self): + flags = {case["graduated"] for case in CASES} + + assert flags == {True, False}, ( + f"graduated cases feed evaluate's regression_cases and held-out " + f"cases its held_out_cases, and evaluate raises on an empty " + f"held_out_cases, so both kinds must exist: {sorted(flags)}" + ) + + def test_held_out_and_graduated_ids_are_disjoint(self): + both = sorted(set(held_out_ids()) & set(graduated_ids())) + + assert both == [], ( + f"a case is a correctness contract or a reading, never both -- " + f"counting it twice moves the mean it also gates: {both}" + ) + + def test_held_out_and_graduated_ids_cover_every_id(self): + covered = set(held_out_ids()) | set(graduated_ids()) + + assert covered == {case["id"] for case in CASES}, ( + f"a case in neither list is a case nothing runs: " + f"{sorted({case['id'] for case in CASES} ^ covered)}" + ) + + def test_the_partition_follows_the_graduated_flag(self): + assert set(graduated_ids()) == { + case["id"] for case in CASES if case["graduated"] is True + } + assert set(held_out_ids()) == { + case["id"] for case in CASES if case["graduated"] is False + } + + def test_the_accessors_return_tuples_of_ids(self): + assert isinstance(held_out_ids(), tuple) + assert isinstance(graduated_ids(), tuple) + assert all(isinstance(case_id, str) for case_id in held_out_ids()) + assert all(isinstance(case_id, str) for case_id in graduated_ids()) + + def test_case_by_id_returns_the_entry_for_every_id(self): + assert [case_by_id(case["id"]) for case in CASES] == list(CASES) + + def test_case_by_id_raises_key_error_for_an_unknown_id(self): + with pytest.raises(KeyError): + case_by_id("nope") + + def test_no_criterion_leaks_into_the_task_the_actor_is_handed(self): + leaked = [ + (case["id"], criterion) + for case in CASES + for criterion in (*case["expect"], *case["forbid"]) + if criterion in case["task"] + ] + + assert leaked == [], ( + f"task is handed to the actor verbatim: a criterion quoted in it " + f"turns the run into an exam the actor can study for, and the " + f"report then measures exam technique: {leaked}" + ) + + def test_the_module_imports_nothing_outside_the_standard_library(self): + outside = sorted(_imported_roots() - sys.stdlib_module_names) + + assert outside == [], ( + f"the case set is the evaluator's only source of truth and must " + f"import on a bare interpreter; scripts/ ships in no wheel and " + f"has no dependencies to declare: {outside}" + ) + + def test_the_module_parses_no_serialisation_format(self): + parsers = sorted(_imported_roots() & _SERIALISATION_MODULES) + + assert parsers == [], ( + f"cases are Python literals: a file format adds a parser, a " + f"schema and a second place to read before anyone can tell what " + f"is asserted: {parsers}" + ) + + def test_the_module_does_not_read_the_environment(self): + tree = ast.parse(_SOURCE.read_text(encoding="utf-8")) + + assert not reads_environment(tree), ( + "an evaluation whose cases depend on the shell is not " + "reproducible; the case set takes no configuration at all" + ) diff --git a/tests/test_harness_catalog_fixture.py b/tests/test_harness_catalog_fixture.py new file mode 100644 index 0000000..0916359 --- /dev/null +++ b/tests/test_harness_catalog_fixture.py @@ -0,0 +1,494 @@ +"""The published harness example, the licence table, and the exit runbook. + +Four documents make a promise this repository has to keep. + +``docs/concepts/harness.example.toml`` shows a reader what a harness catalog +looks like. An example that no longer parses teaches the wrong grammar +confidently, so it is loaded here through the *real* +:func:`molmcp.components.load_harness_catalog` rather than a copy of the +parser. This module deliberately defines no catalog type of its own — +``molmcp.components`` owns the schema, and a second definition would be the +one that drifts. + +``docs/concepts/harness.md`` fences a ``~/.molmcp/settings.json`` snippet whose +``harness`` value is the list of named sources an install may serve from. +``molmcp config harness set`` now writes entries into that same file, but the +snippet is still where a reader is shown the shape — the one a hand-edit has to +produce, and the one the verb leaves behind — so it is held to the same +discipline as the catalog example one paragraph up: parsed as JSON here, and +each entry handed to the real :class:`molmcp.settings.HarnessSource`, so a +snippet that drifts from the type fails the build rather than teaching a shape +nothing accepts. + +``docs/guides/harness-migration.md`` is a runbook a human follows. It stops +before every operation that mutates a repository on GitHub, because each of +those needs its own authorisation; the stop is pinned here so that a later +edit cannot quietly turn a description into an instruction. + +``LICENSE`` is molmcp's grant. The licence table on the concept page describes +it, and must never be read as reissuing it. +""" + +from __future__ import annotations + +import ast +import json +import re +import tomllib +from pathlib import Path + +import pytest + +from molmcp import settings as st +from molmcp.components import ( + CatalogError, + ComponentKind, + load_harness_catalog, +) + +_ROOT = Path(__file__).resolve().parents[1] +_SRC = _ROOT / "src" / "molmcp" +_DOCS = _ROOT / "docs" +_NOTES = _ROOT / ".claude" / "notes" + +_EXAMPLE = _DOCS / "concepts" / "harness.example.toml" +_CONCEPT = _DOCS / "concepts" / "harness.md" +_RUNBOOK = _DOCS / "guides" / "harness-migration.md" +_CONTRACT = _NOTES / "harness-contract.md" +_NOTES_INDEX = _NOTES / "README.md" +_LICENSE = _ROOT / "LICENSE" +_INSTALLATION = _DOCS / "get-started" / "installation.md" +_WORKBENCH = _DOCS / "guides" / "molvis-workbench.md" +_ZENSICAL = _ROOT / "zensical.toml" + +#: Commit identity a caller supplies. Written out here rather than read from +#: the example on purpose: identity lives outside the catalog file, so a test +#: that took it from the file would be asserting the opposite of the rule. +_SHA = "9f1c3b2a7d4e0165c8a9b3d27e5f10486c73ab92" +_CAPABILITIES = frozenset({"provider-sdk", "harness-catalog"}) + +#: The keys the concept page names, spelled out again here so that changing +#: one side fails instead of silently agreeing with itself. +_TOP_LEVEL_KEYS = frozenset({"requires", "component", "component_root"}) +_COMPONENT_KEYS = frozenset({"kind", "name", "path", "entrypoint"}) +_BUNDLE_KEYS = frozenset({"kind", "name", "members", "requires"}) +_ENTRYPOINT_KINDS = frozenset({"provider", "overlay"}) + +#: Vocabulary that belongs to the concept page and nowhere else. +_LABELS = ("official", "gate", "canary") + +#: Remote operations the runbook may only describe *after* it has stopped. +_GITHUB_MUTATIONS = ("create", "archive", "bundle", "delet") + +#: An install line for the repository that is being retired. +_MARKETPLACE_ADD = re.compile(r"marketplace\s+add\s+\S*molcrafts-harness", re.I) + +#: A fenced JSON code block, body only. Markdown is matched rather than parsed +#: because one fence on one page is the whole subject; a Markdown parser would +#: be a dependency taken on to read four lines. +_JSON_FENCE = re.compile(r"^```json\n(.*?)^```", re.M | re.S) + +#: The settings file the concept page teaches a reader to edit by hand, named +#: here so that renaming it on the page fails rather than quietly unpins the +#: snippet below. +_SETTINGS_FILE = "~/.molmcp/settings.json" + +#: The dotted key the ordered source list replaced. ``config set`` exits 2 on +#: it now, so a page still showing it hands the reader a broken command. +_RETIRED_HARNESS_KEY = "harness.owner" + +#: A registration line of the shape an entry-point table uses. +_HARNESS_ENTRY_POINT = re.compile(r"^harness\s*=\s*\S", re.M) + +#: Pages that may only point at the concept page, never restate its contract. +_POINTER_PAGES = ( + _DOCS / "concepts" / "architecture.md", + _DOCS / "concepts" / "provider-design.md", + _DOCS / "concepts" / "providers.md", + _DOCS / "guides" / "write-a-provider.md", + _DOCS / "reference" / "cli.md", + _WORKBENCH, + _INSTALLATION, +) + +_DOCSTRING_OWNERS = ( + ast.Module, + ast.ClassDef, + ast.FunctionDef, + ast.AsyncFunctionDef, +) + + +def _docstring_ids(tree: ast.Module) -> set[int]: + """Identify the string constants that are docstrings rather than code.""" + found: set[int] = set() + for node in ast.walk(tree): + if not isinstance(node, _DOCSTRING_OWNERS) or not node.body: + continue + first = node.body[0] + if not isinstance(first, ast.Expr): + continue + value = first.value + if isinstance(value, ast.Constant) and isinstance(value.value, str): + found.add(id(value)) + return found + + +def _modules_naming_the_catalog_file() -> set[str]: + """Source files that mention ``harness.toml`` in executable code. + + Comments never reach the syntax tree and docstrings are filtered out, so + what remains is the set of modules that actually resolve the filename. + """ + naming: set[str] = set() + for path in sorted(_SRC.rglob("*.py")): + tree = ast.parse(path.read_text(encoding="utf-8")) + skip = _docstring_ids(tree) + for node in ast.walk(tree): + if not isinstance(node, ast.Constant): + continue + if not isinstance(node.value, str) or id(node) in skip: + continue + if "harness.toml" in node.value: + naming.add(path.relative_to(_SRC).as_posix()) + return naming + + +def _markdown_under(*roots: Path) -> list[Path]: + return [path for root in roots for path in sorted(root.rglob("*.md"))] + + +def _nav_targets(node: object) -> list[str]: + if isinstance(node, str): + return [node] + if isinstance(node, list): + return [target for item in node for target in _nav_targets(item)] + if isinstance(node, dict): + return [target for item in node.values() for target in _nav_targets(item)] + return [] + + +def _numbered_headings(text: str) -> list[str]: + return re.findall(r"^##\s*(\d+)\.", text, re.M) + + +def _settings_snippets(text: str) -> list[dict[str, object]]: + """Parse every fenced JSON block on a page that configures ``harness``. + + Selection is by content, not by position: a block qualifies by being a JSON + object with a ``harness`` key. Anchoring on the first fence instead would + make inserting a paragraph above it silently change what is asserted, and + would let a second, drifting copy of the snippet appear unnoticed. + + Args: + text: One Markdown page. + + Returns: + Each qualifying block, parsed, in the order the page fences them. + + Raises: + json.JSONDecodeError: If any ```json block on the page is not JSON. A + fence labelled ``json`` that does not parse is a defect wherever it + sits, so it is reported rather than filtered out. + """ + blocks = [json.loads(body) for body in _JSON_FENCE.findall(text)] + return [b for b in blocks if isinstance(b, dict) and "harness" in b] + + +@pytest.fixture(scope="module") +def example_text() -> str: + return _EXAMPLE.read_text(encoding="utf-8") + + +@pytest.fixture(scope="module") +def example_table(example_text: str) -> dict[str, object]: + return tomllib.loads(example_text) + + +@pytest.fixture(scope="module") +def concept_text() -> str: + return _CONCEPT.read_text(encoding="utf-8") + + +@pytest.fixture(scope="module") +def settings_snippets(concept_text: str) -> list[dict[str, object]]: + """Every ``harness``-bearing JSON block the concept page fences. + + The list is handed over whole rather than unwrapped to a single block, so + that "there is exactly one" is a named assertion in + ``test_concept_page_fences_one_settings_file`` instead of a fixture that + fails before any test runs. + """ + return _settings_snippets(concept_text) + + +@pytest.fixture +def catalog(example_text: str, tmp_path: Path): + """Load the published example under the name a consumer would read. + + The copy is the point. The example is published as + ``harness.example.toml`` and consumed as ``harness.toml``, and + ``load_harness_catalog`` only ever joins the second name onto a root the + caller hands it — never onto the working directory. + """ + (tmp_path / "harness.toml").write_text(example_text, encoding="utf-8") + return load_harness_catalog(tmp_path, _SHA, _CAPABILITIES) + + +class TestHarnessCatalogFixture: + # --------------------------------------------------------------- example + + def test_published_example_loads_through_the_real_loader(self, catalog): + assert catalog.sha == _SHA + assert set(catalog.requires) <= _CAPABILITIES + assert catalog.components + assert {b.name for b in catalog.bundles} == {"sci", "dev"} + + def test_example_carries_every_key_the_page_names(self, example_table): + assert set(example_table) == _TOP_LEVEL_KEYS + rows = example_table["component"] + assert isinstance(rows, list) + assert rows + + component_keys: set[str] = set() + bundle_keys: set[str] = set() + for row in rows: + assert isinstance(row, dict) + if row.get("kind") == "bundle": + assert set(row) <= _BUNDLE_KEYS, row + assert {"kind", "name", "members"} <= set(row), row + bundle_keys |= set(row) + else: + assert set(row) <= _COMPONENT_KEYS, row + assert {"kind", "name", "path"} <= set(row), row + component_keys |= set(row) + + # Every named key is demonstrated at least once, not merely allowed. + assert component_keys == _COMPONENT_KEYS + assert bundle_keys == _BUNDLE_KEYS + + def test_example_demonstrates_every_component_kind(self, catalog): + assert {spec.kind for spec in catalog.components} == set(ComponentKind) + + def test_entrypoint_is_on_exactly_the_kinds_that_need_one(self, catalog): + for spec in catalog.components: + needs = str(spec.kind) in _ENTRYPOINT_KINDS + assert (spec.entrypoint is not None) is needs, spec + + def test_id_is_derived_and_never_written(self, catalog, example_table): + for row in example_table["component"]: + assert "id" not in row, row + for spec in catalog.components: + assert spec.id == f"{spec.kind}.{spec.name}" + assert catalog.get(spec.id) is spec + + def test_identity_and_labels_are_not_catalog_keys(self, example_table): + assert "sha" not in example_table + assert "label" not in example_table + for row in example_table["component"]: + assert "sha" not in row, row + assert "label" not in row, row + + def test_a_label_key_stops_the_file_loading(self, example_text, tmp_path): + """The counter-example: a label cannot be smuggled into the grammar.""" + spiked = f'label = "official"\n{example_text}' + (tmp_path / "harness.toml").write_text(spiked, encoding="utf-8") + with pytest.raises(CatalogError): + load_harness_catalog(tmp_path, _SHA, _CAPABILITIES) + + def test_example_lives_under_docs_and_not_at_the_repo_root(self): + assert _EXAMPLE.is_file() + assert not (_ROOT / "harness.toml").exists() + assert not (_ROOT / "harness.example.toml").exists() + + def test_consumed_filename_is_resolved_in_exactly_one_module(self): + assert _modules_naming_the_catalog_file() == {"components/catalog.py"} + + # ---------------------------------------------------------- concept page + + def test_page_states_the_two_registries_are_disjoint(self): + text = _CONCEPT.read_text(encoding="utf-8") + assert "molmcp.providers" in text + assert "Git SHA" in text + assert "molmcp serve harness" in text + assert "plane id" in text + + def test_page_treats_the_three_words_as_labels_on_a_sha(self): + text = _CONCEPT.read_text(encoding="utf-8") + for label in _LABELS: + assert label in text + assert "not settings" in text + assert "not environment variables" in text + + def test_page_maps_the_example_to_the_consumed_filename(self): + text = _CONCEPT.read_text(encoding="utf-8") + assert "harness.example.toml" in text + assert "harness.toml" in text + assert "working directory" in text + + def test_page_refuses_wikiskill_as_an_init_channel(self): + text = _CONCEPT.read_text(encoding="utf-8") + assert "WikiSkill" in text + for wrapped in ("packages", "molvis_open", "molq_*", "molexp_*"): + assert wrapped in text + + def test_page_states_a_new_empty_repo_not_a_rename(self): + text = _CONCEPT.read_text(encoding="utf-8") + assert "MolCrafts/harness" in text + assert "molcrafts-harness" in text + # The page says "a new, empty repository"; match the claim, not one + # particular way of punctuating it. + assert re.search(r"new,?\s+empty\s+repository", text) is not None + assert "rename" in text + + # ------------------------------------------------- the settings-file shape + + def test_neither_page_names_the_retired_dotted_harness_key(self): + """``harness`` is a list now, so the dotted key addresses nothing. + + ``_SCHEMA["harness"]`` is ``list``, which makes ``_resolve`` refuse + every ``harness.`` path, so ``molmcp config set harness.owner`` + exits 2. A page still showing it would be handing the reader a command + that cannot work. + """ + for path in (_CONCEPT, _INSTALLATION): + assert _RETIRED_HARNESS_KEY not in path.read_text(encoding="utf-8"), path + + def test_concept_page_fences_one_settings_file( + self, concept_text, settings_snippets + ): + """One snippet, and the page says which file it is. + + ``config harness set`` writes into that file rather than standing in + for it — a file that already fails validation on read is one the verb + cannot load either, and still has to be opened — so the page has to + say which file it is. Exactly one snippet, because two would be two + copies of a contract and one of them would be the stale one. + """ + assert _SETTINGS_FILE in concept_text + assert len(settings_snippets) == 1, settings_snippets + + def test_snippet_gives_harness_a_list_of_entry_objects(self, settings_snippets): + """The shape claim: a list of objects, keyed like the dataclass. + + ``_HARNESS_ENTRY_KEYS`` is derived from + :class:`molmcp.settings.HarnessSource` rather than written out, here + and in ``settings.py`` alike, so a fifth field added to the type widens + both sides at once. + """ + entries = settings_snippets[0]["harness"] + assert isinstance(entries, list) + assert entries, "an empty list would demonstrate nothing" + for entry in entries: + assert isinstance(entry, dict), entry + assert set(entry) <= st._HARNESS_ENTRY_KEYS, entry + + def test_every_snippet_entry_constructs_a_harness_source(self, settings_snippets): + """The type is the judge, exactly as the loader would be. + + Re-stating the entry rules here would create a second definition of + them, and it would be this one that drifted. The snippet is instead + handed to the real type, so a doc example that stops being loadable + fails the build. + """ + for entry in settings_snippets[0]["harness"]: + source = st.HarnessSource(**entry) + assert source.name + + # --------------------------------------------------------------- licence + + def test_root_license_is_still_bsd_3_clause(self): + text = _LICENSE.read_text(encoding="utf-8") + assert text.startswith("BSD 3-Clause License") + + def test_license_table_records_the_grant_without_reissuing_it(self): + text = _CONCEPT.read_text(encoding="utf-8") + assert "BSD-3-Clause" in text + assert "MIT" in text + assert "LICENSE" in text + + # --------------------------------------------------------------- runbook + + def test_runbook_is_five_numbered_steps(self): + text = _RUNBOOK.read_text(encoding="utf-8") + assert _numbered_headings(text) == ["1", "2", "3", "4", "5"] + + def test_runbook_stops_before_any_github_mutation(self): + text = _RUNBOOK.read_text(encoding="utf-8") + stop = text.index("STOP") + lowered = text.lower() + for word in _GITHUB_MUTATIONS: + first = lowered.find(word) + # A word the runbook never uses cannot appear too early. find() + # answers -1 for absent, which is not "before the STOP". + if first == -1: + continue + assert first > stop, ( + f"{word!r} is at {first}, before the STOP at {stop}; the " + "runbook may only describe remote mutations after it stops" + ) + + def test_runbook_forbids_piling_provider_repos_into_the_new_one(self): + text = _RUNBOOK.read_text(encoding="utf-8") + assert "do not pile provider" in text.lower() + + # ---------------------------------------------------------- notes and nav + + def test_contract_note_holds_the_two_rules_and_no_schema(self): + text = _CONTRACT.read_text(encoding="utf-8") + assert "new empty repository" in text + assert "molcrafts-harness" in text + assert "after cutover" in text + assert "Git SHA" in text + for schema_word in ("[[component]]", "entrypoint", "members", "BSD"): + assert schema_word not in text, schema_word + + def test_contract_note_is_indexed(self): + assert "harness-contract.md" in _NOTES_INDEX.read_text(encoding="utf-8") + + def test_nav_lists_the_concept_page_and_the_runbook(self): + site = tomllib.loads(_ZENSICAL.read_text(encoding="utf-8")) + targets = _nav_targets(site["project"]["nav"]) + assert "concepts/harness.md" in targets + assert "guides/harness-migration.md" in targets + + # --------------------------------------------------------- pointer pages + + def test_every_pointer_page_links_to_the_concept_page(self): + for path in _POINTER_PAGES: + text = path.read_text(encoding="utf-8") + assert "harness.md)" in text, path + + def test_pointer_pages_add_no_entry_point_and_no_label_words(self): + for path in _POINTER_PAGES: + text = path.read_text(encoding="utf-8") + assert not _HARNESS_ENTRY_POINT.search(text), path + assert "canary" not in text, path + + def test_workbench_separates_its_playbook_from_the_sha_catalog(self): + text = _WORKBENCH.read_text(encoding="utf-8") + assert "molvis-agent-e2e/" in text + assert "Git SHA plugin catalog" in text + assert "../concepts/harness.md" in text + + def test_installation_keeps_its_uv_prerelease_warning(self): + text = _INSTALLATION.read_text(encoding="utf-8") + assert "Without `--prerelease=allow`, uv will not install 0.6+" in text + assert "4.0.0b5" in text + + # ------------------------------------------------------- retired address + + def test_no_page_advertises_the_old_marketplace_as_current(self): + offenders = [ + path.relative_to(_ROOT).as_posix() + for path in _markdown_under(_DOCS, _NOTES) + if _MARKETPLACE_ADD.search(path.read_text(encoding="utf-8")) + ] + assert offenders == [] + + def test_new_pages_introduce_no_environment_variable(self): + for path in (_EXAMPLE, _RUNBOOK, _CONTRACT): + assert "MOLMCP_" not in path.read_text(encoding="utf-8"), path + for line in _CONCEPT.read_text(encoding="utf-8").splitlines(): + if "MOLMCP_" in line: + assert "reads no" in line, line diff --git a/tests/test_harness_eval.py b/tests/test_harness_eval.py new file mode 100644 index 0000000..d388df5 --- /dev/null +++ b/tests/test_harness_eval.py @@ -0,0 +1,757 @@ +"""The observation adapter: a blind transcript reading turned into a verdict. + +Mirrors ``scripts/harness_eval.py`` — the one thin entry that hands an +observer subagent's structured output to the already-shipped +:func:`molmcp.evolution.evaluate`. ``scripts/`` is flat rather than a +package, so the mirrored unit path is this single ``tests/`` module; one +class per behaviour the adapter owns, and nothing here starts an agent, +a host or a process. + +Four disciplines are pinned here that no single assertion makes obvious. + +*The manifest unblinds, the observer never does.* The observation carries +only the blind labels ``A`` / ``B``; which one is the challenger comes +from ``manifest["sides"]``. An observation that so much as names a side +is refused before the store is touched, and the positive proof is +``test_swapping_the_sides_flips_the_verdict``: one observation read twice +under swapped manifests must come out ``ACCEPTED`` one way and +``WORSE_CALL_COUNT`` the other. A reader who assigned sides from the +payload would get the same verdict twice. + +*Giving up must not read cheap.* An unfinished round makes fewer calls +and fewer errors, so averaging a held-out reading with ``contract_met`` +false in would make abandonment look like a gain. The guard test builds +exactly that shape — the abandoned cell also carries the lowest +``call_count`` in the whole observation — and the negative control flips +that one field to true and gets ``ACCEPTED``, which is precisely the +false win the refusal exists to stop. + +*Two readings cannot be read off a transcript.* ``tokens`` and +``latency_s`` are refused on the way in and pinned to zero on the way +out: under ``evaluate``'s independent comparisons, 0 against 0 is the +only value that neither convicts nor acquits. Permitting the observer to +write them invites the next one to guess a number. + +*The readings are sums of held-out cases only.* A graduated case is a +correctness contract, not a reading; its counts are loud here (a +``call_count`` of ``_GRADUATED_CALL_COUNT``) so an implementation that +summed them in cannot land on the expected number by luck. + +The case ids come from ``harness_cases.CASES`` rather than literals: the +suite is data that may still grow, and only the deliberately unknown id +is spelled out. Everything outbound is a fake — a store that records its +``tree_path`` calls, a spy that stands in for ``ObservedReplay`` and +counts every dispatch. ``report`` must build its replay by looking up the +module-global ``ObservedReplay``, which is what lets the short-circuit +test prove zero calls. No network, no git, no subprocess, no environment +variable, and the trees are literal paths that are never created. +""" + +from __future__ import annotations + +import ast +import dataclasses +import inspect +import re +from collections.abc import Mapping, Sequence +from pathlib import Path + +import harness_eval +import pytest +from harness_cases import CASES +from harness_eval import ( + ObservedChallenger, + ObservedReplay, + ObservedRunner, + main, + report, +) + +from molmcp.components import UnknownShaError +from molmcp.evolution import ( + ACCEPTED, + REGRESSION_FAILED, + WORSE_CALL_COUNT, + EvalCase, + EvaluationError, + EvaluationReport, + Metrics, +) + +_REPO = Path(__file__).resolve().parents[1] +_SOURCE = _REPO / "scripts" / "harness_eval.py" + +#: Read from the case set, never spelled out: which ids are held-out and +#: which have graduated is the case module's decision, not this module's. +_HELD_OUT_IDS: tuple[str, ...] = tuple( + str(case["id"]) for case in CASES if not case["graduated"] +) +_GRADUATED_IDS: tuple[str, ...] = tuple( + str(case["id"]) for case in CASES if case["graduated"] +) + +#: How many held-out cases one side's per-seed reading sums over. +_HELD_OUT_COUNT = len(_HELD_OUT_IDS) + +#: The one id that must *not* resolve. A typo silently averaged into the +#: mean is worse than a refusal, so this is the only literal id here. +_UNKNOWN_CASE_ID = "no-such-case" + +#: The blind labels the observer is allowed to use, and nothing else. +_LABELS: tuple[str, ...] = ("A", "B") + +_SCHEMA = "harness-eval/1" + +#: Full shas: the manifest carries complete 40-character strings. +_CHAMPION_SHA = "a" * 40 +_CHALLENGER_SHA = "b" * 40 +_COMPONENT = "daily-pack-skill" +_AFFECTED_PATHS: tuple[str, ...] = ("skills/daily/pack.md",) + +#: Never created on disk. The store hands them over, ``evaluate`` passes +#: them to the seams, and nothing stats them. +_CHAMPION_TREE = Path("/published/champion/tree") +_CHALLENGER_TREE = Path("/published/challenger/tree") +_TREES: Mapping[str, Path] = { + _CHAMPION_SHA: _CHAMPION_TREE, + _CHALLENGER_SHA: _CHALLENGER_TREE, +} + +#: Repeat rounds, not random seeds: the same prompt run three times. +_SEEDS: tuple[int, ...] = (1, 2, 3) + +#: Two rounds whose numbers are not ``DEFAULT_SEEDS``, so a report that +#: echoes them back cannot have fallen through to the default. +_ODD_SEEDS: tuple[int, ...] = (2, 5) + +_SIDES: Mapping[str, str] = {"A": "champion", "B": "challenger"} +_SIDES_SWAPPED: Mapping[str, str] = {"A": "challenger", "B": "champion"} + +#: Per held-out reading. One side cheaper than the other by one call per +#: case is the whole difference in the accepted fixture. +_TIED_CALLS = 6 +_CHEAPER_CALLS = 5 +_TIED_ERRORS = 1 + +#: What an abandoned round reads like: the cheapest cell in the payload. +_ABANDONED_CALLS = 1 + +#: Graduated rows are loud on purpose. Summing them into a reading would +#: move the reported mean by hundreds, not by a rounding step. +_GRADUATED_CALL_COUNT = 1000 +_GRADUATED_TOOL_ERRORS = 50 + +#: The two per-seed held-out counts of the seeds fixture, and the mean +#: written out on its own rather than derived from them. +_SLOW_SEED_CALLS = 8 +_MEAN_CALLS = 7 +_FEW_ERRORS = 2 +_MANY_ERRORS = 4 +_MEAN_ERRORS = 3 + +#: Distinct readings for the dispatch test: whichever table +#: ``ObservedReplay`` picks is visible in the numbers it returns. +_CHAMPION_READING = Metrics(tool_errors=3, call_count=13, tokens=0, latency_s=0.0) +_CHALLENGER_READING = Metrics(tool_errors=1, call_count=7, tokens=0, latency_s=0.0) +_CHAMPION_TABLE: Mapping[int, Metrics] = {seed: _CHAMPION_READING for seed in _SEEDS} +_CHALLENGER_TABLE: Mapping[int, Metrics] = { + seed: _CHALLENGER_READING for seed in _SEEDS +} +_REPLAY_CASES: tuple[EvalCase, ...] = tuple( + EvalCase(id=case_id) for case_id in _HELD_OUT_IDS +) + +#: The seven frozen reason literals in their *quoted* form. A bare scan +#: for ``accepted`` would ban ``report.accepted``, which is a field read, +#: and push a legitimate ``main`` into ``getattr`` to pass this check. +_REASON_WORDS: tuple[str, ...] = ( + "accepted", + "worse_tool_errors", + "worse_call_count", + "worse_tokens", + "worse_latency", + "no_practical_gain", + "regression_failed", +) +_QUOTED_REASONS: tuple[str, ...] = tuple(f'"{word}"' for word in _REASON_WORDS) + tuple( + f"'{word}'" for word in _REASON_WORDS +) + +#: A second threshold, a second telemetry source, or a second way to get +#: a tree. Each one would make two runs of this evaluator incomparable. +_FORBIDDEN_FRAGMENTS: tuple[str, ...] = ( + "DROP_", + "os.environ", + "getenv", + "anthropic", + "subprocess", + "shutil", + "tarfile", +) + +#: Scanned as words, not substrings: ``digit`` must not read as ``git`` +#: and ``underscore`` must not read as ``score``. +_FORBIDDEN_WORDS: tuple[str, ...] = (r"\bgit\b", r"(?i)\bscores?\b") + +#: The three flags ``main`` must require, none of them defaulted. +_MAIN_FLAGS: tuple[str, ...] = ("--observation", "--manifest", "--store-root") + + +class FakeStore: + """Stand-in for ``ImmutableGitStore`` — one table lookup, recorded. + + Deliberately not a subclass: the adapter reads exactly one method, + and a fake that inherited the real store would hide a rename of it. + """ + + def __init__(self, trees: Mapping[str, Path]) -> None: + self._trees = dict(trees) + self.calls: list[str] = [] + + def tree_path(self, sha: str) -> Path: + self.calls.append(sha) + if sha not in self._trees: + raise UnknownShaError(sha) + return self._trees[sha] + + +class SpyReplay: + """One ``ObservedReplay`` wrapped so every dispatch is recorded.""" + + def __init__( + self, + inner: ObservedReplay, + calls: list[tuple[str | Path, tuple[str, ...], int]], + ) -> None: + self._inner = inner + self._calls = calls + + def __call__( + self, target: str | Path, cases: Sequence[EvalCase], seed: int + ) -> Metrics: + self._calls.append((target, tuple(case.id for case in cases), seed)) + return self._inner(target, cases, seed) + + @property + def calls(self) -> list[tuple[str | Path, tuple[str, ...], int]]: + return list(self._calls) + + +class SpyReplayFactory: + """Stands in for the ``ObservedReplay`` class inside ``report``. + + Every instance it hands out delegates to the real ``ObservedReplay`` + and appends to one shared ``calls`` list, so a test can prove the + replay was never reached at all. + """ + + def __init__(self) -> None: + self.calls: list[tuple[str | Path, tuple[str, ...], int]] = [] + + def __call__(self, *args: object, **kwargs: object) -> SpyReplay: + return SpyReplay(ObservedReplay(*args, **kwargs), self.calls) + + +def _per_seed(value: Mapping[int, int] | int, seeds: Sequence[int]) -> dict[int, int]: + """One count per seed, from either a table or a single number.""" + if isinstance(value, Mapping): + return {seed: value[seed] for seed in seeds} + return {seed: value for seed in seeds} + + +def _reading( + case_id: str, + seed: int, + side: str, + *, + contract_met: bool = True, + tool_errors: int = 0, + call_count: int = _TIED_CALLS, +) -> dict[str, object]: + """One observer row: a blind side, a round, and what it counted.""" + return { + "case_id": case_id, + "seed": seed, + "side": side, + "contract_met": contract_met, + "tool_errors": tool_errors, + "call_count": call_count, + } + + +def _readings( + *, + calls: Mapping[str, Mapping[int, int] | int], + errors: Mapping[str, Mapping[int, int] | int] | None = None, + seeds: Sequence[int] = _SEEDS, + failed: frozenset[tuple[str, int, str]] = frozenset(), +) -> list[dict[str, object]]: + """The complete grid: every ``(label, seed, case)`` cell exactly once. + + ``calls`` and ``errors`` are per *held-out reading*, so one side's + reading for one seed is that number times the held-out case count. + Graduated rows carry the loud counts on both sides — the observer + does not know which case graduated either. + """ + errors = {label: 0 for label in _LABELS} if errors is None else errors + rows: list[dict[str, object]] = [] + for label in _LABELS: + call_table = _per_seed(calls[label], seeds) + error_table = _per_seed(errors[label], seeds) + for seed in seeds: + for case_id in _HELD_OUT_IDS: + rows.append( + _reading( + case_id, + seed, + label, + contract_met=(label, seed, case_id) not in failed, + tool_errors=error_table[seed], + call_count=call_table[seed], + ) + ) + for case_id in _GRADUATED_IDS: + rows.append( + _reading( + case_id, + seed, + label, + contract_met=(label, seed, case_id) not in failed, + tool_errors=_GRADUATED_TOOL_ERRORS, + call_count=_GRADUATED_CALL_COUNT, + ) + ) + return rows + + +def _tied_readings(**kwargs: object) -> list[dict[str, object]]: + """Both labels reading exactly the same, so one edit decides.""" + return _readings(calls={label: _TIED_CALLS for label in _LABELS}, **kwargs) + + +def _cheaper_on_b( + *, failed: frozenset[tuple[str, int, str]] = frozenset() +) -> list[dict[str, object]]: + """Label ``B`` one call per case cheaper, everything else tied.""" + return _readings( + calls={"A": _TIED_CALLS, "B": _CHEAPER_CALLS}, + errors={label: _TIED_ERRORS for label in _LABELS}, + failed=failed, + ) + + +def _with_cell( + rows: Sequence[Mapping[str, object]], + *, + case_id: str, + seed: int, + side: str, + **fields: object, +) -> list[dict[str, object]]: + """A copy of *rows* with one cell's fields replaced; nothing mutated.""" + cell = (case_id, seed, side) + return [ + {**row, **fields} + if (row["case_id"], row["seed"], row["side"]) == cell + else dict(row) + for row in rows + ] + + +def _observation( + rows: Sequence[Mapping[str, object]], **extra: object +) -> dict[str, object]: + """What the observer writes: a schema tag, rows, and no side names.""" + return {"schema": _SCHEMA, "readings": [dict(row) for row in rows], **extra} + + +def _manifest(**overrides: object) -> dict[str, object]: + """What the orchestrator wrote *before* the run, and never showed.""" + manifest: dict[str, object] = { + "champion_sha": _CHAMPION_SHA, + "challenger_sha": _CHALLENGER_SHA, + "component": _COMPONENT, + "affected_paths": list(_AFFECTED_PATHS), + "seeds": list(_SEEDS), + "sides": dict(_SIDES), + } + return {**manifest, **overrides} + + +def _store(published: Mapping[str, Path] | None = None) -> FakeStore: + """Both shas published unless a test says one of them is not.""" + return FakeStore(_TREES if published is None else published) + + +def _source() -> str: + assert _SOURCE.is_file(), f"{_SOURCE} does not exist yet" + return _SOURCE.read_text(encoding="utf-8") + + +def _import_pairs(tree: ast.Module) -> set[tuple[str, str]]: + """Every ``(module, name)`` the source imports with ``from``.""" + return { + (node.module or "", alias.name) + for node in ast.walk(tree) + if isinstance(node, ast.ImportFrom) + for alias in node.names + } + + +#: Observation keys that only an unblinded observer could have written. +_LEAKED_KEYS = ( + pytest.param("sides", dict(_SIDES), id="sides"), + pytest.param("champion", "A", id="champion"), + pytest.param("challenger", "B", id="challenger"), + pytest.param("champion_sha", _CHAMPION_SHA, id="champion_sha"), + pytest.param("challenger_sha", _CHALLENGER_SHA, id="challenger_sha"), +) + +#: ``sides`` maps that are not a bijection onto the two roles. Each one +#: leaves at least one reading with no side, or two readings with one. +_BROKEN_SIDES = ( + pytest.param({"A": "champion", "B": "champion"}, id="both-champion"), + pytest.param({"A": "challenger", "B": "challenger"}, id="both-challenger"), + pytest.param({"A": "champion"}, id="missing-a-side"), + pytest.param( + {"A": "champion", "B": "challenger", "C": "champion"}, + id="a-third-label", + ), + pytest.param({"A": "champion", "B": "observer"}, id="a-third-role"), +) + +#: Readings the transcript cannot support. Permitting either invites the +#: next observer to guess a number and call it telemetry. +_UNOBSERVABLE = ( + pytest.param("tokens", 900, id="tokens"), + pytest.param("latency_s", 12.5, id="latency_s"), +) + + +class TestReport: + def test_a_cheaper_challenger_is_accepted(self) -> None: + result = report(_observation(_cheaper_on_b()), _manifest(), store=_store()) + + assert result.accepted is True + assert result.reason == ACCEPTED + assert result.candidate_sha == _CHALLENGER_SHA + assert result.champion_sha == _CHAMPION_SHA + assert result.regression_passed is True + + def test_the_seeds_are_recorded_as_the_manifest_gave_them(self) -> None: + """Two rounds that are not ``DEFAULT_SEEDS``, echoed back in order.""" + rows = _readings( + calls={"A": _TIED_CALLS, "B": _CHEAPER_CALLS}, + seeds=_ODD_SEEDS, + ) + + result = report( + _observation(rows), + _manifest(seeds=list(_ODD_SEEDS)), + store=_store(), + ) + + assert result.seeds == _ODD_SEEDS + + def test_a_reading_sums_that_rounds_held_out_cases(self) -> None: + """Per ``(side, seed)``: the sum over held-out cases, then the mean. + + The champion reads 6 calls per case in one round and 8 in the + other, so its mean is 7 per case; an implementation that averaged + the cases instead of summing them would report 7, not 7 times the + held-out count. + """ + rows = _readings( + calls={"A": {2: _TIED_CALLS, 5: _SLOW_SEED_CALLS}, "B": _CHEAPER_CALLS}, + errors={"A": {2: _FEW_ERRORS, 5: _MANY_ERRORS}, "B": _MEAN_ERRORS}, + seeds=_ODD_SEEDS, + ) + + result = report( + _observation(rows), + _manifest(seeds=list(_ODD_SEEDS)), + store=_store(), + ) + + assert result.champion_metrics.call_count == _MEAN_CALLS * _HELD_OUT_COUNT + assert result.champion_metrics.tool_errors == _MEAN_ERRORS * _HELD_OUT_COUNT + assert result.challenger_metrics.call_count == _CHEAPER_CALLS * _HELD_OUT_COUNT + assert result.challenger_metrics.tool_errors == _MEAN_ERRORS * _HELD_OUT_COUNT + + def test_a_graduated_case_never_reaches_the_readings(self) -> None: + """Its counts are loud; a reading that summed them in shows it.""" + result = report(_observation(_cheaper_on_b()), _manifest(), store=_store()) + + for metrics in (result.champion_metrics, result.challenger_metrics): + assert metrics.call_count < _GRADUATED_CALL_COUNT + assert metrics.tool_errors < _GRADUATED_TOOL_ERRORS + + def test_a_failed_graduated_case_short_circuits_before_any_replay( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + """The challenger is strictly cheaper here, so skipping the + contract would accept it rather than reject it.""" + factory = SpyReplayFactory() + monkeypatch.setattr(harness_eval, "ObservedReplay", factory) + failed = frozenset({("B", _SEEDS[0], _GRADUATED_IDS[0])}) + + result = report( + _observation(_cheaper_on_b(failed=failed)), + _manifest(), + store=_store(), + ) + + assert result.reason == REGRESSION_FAILED + assert result.accepted is False + assert result.regression_passed is False + for metrics in (result.champion_metrics, result.challenger_metrics): + assert metrics.tool_errors == 0 + assert metrics.call_count == 0 + assert metrics.tokens == 0 + assert metrics.latency_s == 0.0 + assert factory.calls == [] + + def test_a_failed_graduated_case_on_the_champion_side_is_discarded( + self, + ) -> None: + """Both sides run the graduated case; only the challenger's counts. + + ``evaluate`` runs the contract on the challenger tree alone, so a + champion-side failure must not reject anything. + """ + failed = frozenset({("A", _SEEDS[0], _GRADUATED_IDS[0])}) + + result = report( + _observation(_cheaper_on_b(failed=failed)), + _manifest(), + store=_store(), + ) + + assert result.reason == ACCEPTED + assert result.regression_passed is True + + def test_an_unknown_case_id_is_refused_by_name(self) -> None: + rows = [*_cheaper_on_b(), _reading(_UNKNOWN_CASE_ID, _SEEDS[0], "A")] + + with pytest.raises(EvaluationError, match=_UNKNOWN_CASE_ID): + report(_observation(rows), _manifest(), store=_store()) + + def test_a_missing_cell_is_refused(self) -> None: + """One cell short silently changes the denominator of the mean.""" + rows = _cheaper_on_b() + + with pytest.raises(EvaluationError): + report(_observation(rows[1:]), _manifest(), store=_store()) + + def test_a_duplicated_cell_is_refused(self) -> None: + rows = _cheaper_on_b() + + with pytest.raises(EvaluationError): + report( + _observation([*rows, dict(rows[0])]), + _manifest(), + store=_store(), + ) + + +class TestBlindnessGuard: + @pytest.mark.parametrize(("key", "value"), _LEAKED_KEYS) + def test_an_observation_that_names_a_side_is_refused( + self, key: str, value: object + ) -> None: + """An observer that can name a side was told which one it was.""" + store = _store() + + with pytest.raises(EvaluationError): + report( + _observation(_cheaper_on_b(), **{key: value}), + _manifest(), + store=store, + ) + + assert store.calls == [] + + @pytest.mark.parametrize("sides", _BROKEN_SIDES) + def test_sides_must_be_a_bijection_onto_the_two_roles( + self, sides: Mapping[str, str] + ) -> None: + with pytest.raises(EvaluationError): + report( + _observation(_cheaper_on_b()), + _manifest(sides=dict(sides)), + store=_store(), + ) + + def test_swapping_the_sides_flips_the_verdict(self) -> None: + """The positive proof: the manifest assigns sides, not the reader. + + One observation, read twice. With ``B`` as the challenger the + cheaper side is the challenger and the report accepts; with the + manifest swapped the very same numbers are a regression. + """ + observation = _observation(_cheaper_on_b()) + + accepted = report(observation, _manifest(sides=dict(_SIDES)), store=_store()) + rejected = report( + observation, _manifest(sides=dict(_SIDES_SWAPPED)), store=_store() + ) + + assert accepted.reason == ACCEPTED + assert accepted.accepted is True + assert rejected.reason == WORSE_CALL_COUNT + assert rejected.accepted is False + + def test_an_abandoned_held_out_run_is_refused_by_case_side_and_seed( + self, + ) -> None: + """The cheapest cell in the payload is the one that gave up.""" + rows = _with_cell( + _tied_readings(), + case_id=_HELD_OUT_IDS[0], + seed=_SEEDS[1], + side="B", + contract_met=False, + call_count=_ABANDONED_CALLS, + ) + + with pytest.raises(EvaluationError) as excinfo: + report(_observation(rows), _manifest(), store=_store()) + + message = str(excinfo.value) + assert _HELD_OUT_IDS[0] in message + assert str(_SEEDS[1]) in message + assert re.search(r"\bB\b|challenger", message), ( + f"the refusal must name the side it read, blind label or " + f"unblinded role; got {message!r}" + ) + + def test_the_same_input_reports_once_that_run_finished(self) -> None: + """The negative control for the refusal above. + + One field differs: the abandoned round is marked finished. Its + single call now reads as the cheapest round anyone ran, and the + verdict is ``ACCEPTED`` — the false win the refusal prevents. + """ + rows = _with_cell( + _tied_readings(), + case_id=_HELD_OUT_IDS[0], + seed=_SEEDS[1], + side="B", + contract_met=True, + call_count=_ABANDONED_CALLS, + ) + + result = report(_observation(rows), _manifest(), store=_store()) + + assert isinstance(result, EvaluationReport) + assert result.reason == ACCEPTED + + +class TestObservedSeams: + @pytest.mark.parametrize(("key", "value"), _UNOBSERVABLE) + def test_a_reading_the_transcript_cannot_carry_is_refused( + self, key: str, value: object + ) -> None: + rows = _cheaper_on_b() + rows[0] = {**rows[0], key: value} + + with pytest.raises(EvaluationError): + report(_observation(rows), _manifest(), store=_store()) + + def test_both_sides_read_zero_tokens_and_zero_latency(self) -> None: + """0 against 0 is the only pair that decides nothing at all.""" + result = report(_observation(_cheaper_on_b()), _manifest(), store=_store()) + + for metrics in (result.champion_metrics, result.challenger_metrics): + assert metrics.tokens == 0 + assert metrics.latency_s == 0.0 + + def test_replay_reads_the_champion_table_for_a_str_target(self) -> None: + """``ReplayFn``'s frozen convention: the champion arrives as a sha.""" + replay = ObservedReplay(_CHAMPION_TABLE, _CHALLENGER_TABLE) + + assert replay(_CHAMPION_SHA, _REPLAY_CASES, _SEEDS[0]) == _CHAMPION_READING + + def test_replay_reads_the_challenger_table_for_a_path_target(self) -> None: + """And the challenger as a tree someone already checked out.""" + replay = ObservedReplay(_CHAMPION_TABLE, _CHALLENGER_TABLE) + + assert replay(_CHALLENGER_TREE, _REPLAY_CASES, _SEEDS[0]) == _CHALLENGER_READING + + def test_both_shas_are_resolved_through_the_store(self) -> None: + store = _store() + + report(_observation(_cheaper_on_b()), _manifest(), store=store) + + assert set(store.calls) == {_CHAMPION_SHA, _CHALLENGER_SHA} + + @pytest.mark.parametrize("missing", ["champion_sha", "challenger_sha"]) + def test_an_unpublished_sha_propagates_rather_than_being_swallowed( + self, missing: str + ) -> None: + """A report on an unpublished tree could never be reproduced.""" + manifest = _manifest() + published = { + sha: tree for sha, tree in _TREES.items() if sha != manifest[missing] + } + + with pytest.raises(UnknownShaError): + report( + _observation(_cheaper_on_b()), + manifest, + store=_store(published), + ) + + def test_the_challenger_carries_the_three_protocol_names(self) -> None: + names = tuple(field.name for field in dataclasses.fields(ObservedChallenger)) + + assert names == ("sha", "component", "affected_paths") + + def test_the_challenger_is_frozen(self) -> None: + challenger = ObservedChallenger( + sha=_CHALLENGER_SHA, + component=_COMPONENT, + affected_paths=_AFFECTED_PATHS, + ) + + with pytest.raises(dataclasses.FrozenInstanceError): + challenger.sha = _CHAMPION_SHA # type: ignore[misc] + + def test_the_runner_keeps_the_contract_runner_signature(self) -> None: + params = tuple(inspect.signature(ObservedRunner.run).parameters) + + assert params == ("self", "tree", "cases") + + def test_main_takes_argv_and_defaults_to_nothing_else(self) -> None: + params = inspect.signature(main).parameters + + assert tuple(params) == ("argv",) + assert params["argv"].default is None + + @pytest.mark.parametrize("flag", _MAIN_FLAGS) + def test_main_names_all_three_paths_on_the_command_line(self, flag: str) -> None: + assert flag in _source() + + @pytest.mark.parametrize("literal", _QUOTED_REASONS) + def test_the_source_copies_no_reason_literal(self, literal: str) -> None: + """A local copy of a reason is a second comparator in waiting.""" + assert literal not in _source() + + @pytest.mark.parametrize("fragment", _FORBIDDEN_FRAGMENTS) + def test_the_source_carries_no_second_mechanism(self, fragment: str) -> None: + """No threshold, no model call, no environment, no second checkout.""" + assert fragment not in _source() + + @pytest.mark.parametrize("pattern", _FORBIDDEN_WORDS) + def test_the_source_names_no_tool_of_its_own(self, pattern: str) -> None: + found = re.search(pattern, _source()) + + assert found is None, ( + f"{pattern} appears in {_SOURCE.name}: the tree comes from the " + f"store and the verdict from molmcp.evolution" + ) + + def test_the_verdict_comes_from_the_upstream_gate(self) -> None: + pairs = _import_pairs(ast.parse(_source())) + + assert ("molmcp.evolution", "evaluate") in pairs diff --git a/tests/test_harness_install.py b/tests/test_harness_install.py new file mode 100644 index 0000000..5ab6bf4 --- /dev/null +++ b/tests/test_harness_install.py @@ -0,0 +1,799 @@ +"""`molmcp init` installs what the *activated* harness commit declares. + +Mirrors ``src/molmcp/harness_install.py``, the missing link of the chain the +last three changes built. ``molmcp config harness set`` registers a source, +``molmcp harness sync`` publishes its ``HEAD`` and promotes that source's +activation pointer, and ``molmcp.host.place_components`` places +``ComponentFile`` rows by kind — but nothing turns a *pointer* into those +rows, so an operator who has synced a harness and run ``molmcp init`` gets +none of it. + +The resolver is what runs in between, and its four obligations are what this +file pins: + +* read each configured source's activation pointer for its ``current`` SHA, + and **skip** a source that has none — a configured source is not a synced + one, and the operator who has not synced yet is not misconfigured; +* load ``harness.toml`` from that commit's tree; +* keep the non-bundle rows, strip ``KIND_PATH_PREFIX`` off each ``path`` for + ``relative``, and join ``component_root`` for the absolute ``source``; +* resolve every row **under its own source's root**, so a multi-source + install never reads one source's components out of another's tree. + +Its own module rather than more of ``tests/test_cli_harness.py``, following +the split already in this suite: that file mirrors ``harness_sync.py``, the +*write* half (fetch, publish, activate), and this one mirrors the *read* half +that ``molmcp init`` composes. The two halves meet on disk here and nowhere +else, which is why nothing is faked between them: the checkout is built by +``git init``, the commit is published by the real ``molmcp harness sync``, +and the pointer is the real file the resolver binds. A seam standing in for +either would keep passing while the two commands disagreed about where a +commit lives. + +**No network.** Every repository here is built under ``tmp_path`` and every +source is a local one, so the local transport is the only one constructed and +it opens no socket. + +``Path.home`` is pinned to the ``home`` fixture, so every destination is the +real host layout without touching the developer's own home. No environment +variable is read: ``tests/test_no_env_switches.py`` scans every module under +``src/molmcp`` for that already. +""" + +from __future__ import annotations + +import ast +import subprocess +from pathlib import Path + +import pytest + +from molmcp import cli +from molmcp import settings as st +from molmcp.host import SKIP_MANAGED_USAGE_SKILL, SKIP_NO_HOST_DESTINATION + +#: Production module this file mirrors, read as text by the isolation tests. +SRC = Path(__file__).resolve().parents[1] / "src" / "molmcp" +RESOLVER_SOURCE = SRC / "harness_install.py" + +#: Dotted module paths the resolver must not reach, and why each one is here. +#: ``molmcp.harness`` carries a module-level +#: ``from .provider_worker.worker import WorkerProvider``, so importing it +#: drags the whole FastMCP-bearing worker stack into the importing process. +#: ``molmcp init`` mounts no plane and must not pay for one, so the resolver +#: reaches ``molmcp.components`` — the stdlib leaf that owns ``Activation``, +#: ``ImmutableGitStore`` and ``load_harness_catalog`` — directly instead of +#: inheriting the cost through the serve-side reader. +FORBIDDEN_IMPORTS: tuple[str, ...] = ("molmcp.harness", "molmcp.provider_worker") + +#: The leaf the resolver is expected to reach instead. +REQUIRED_IMPORT = "molmcp.components" + +#: The host every behavioural test wires. One host, not four: which directory +#: a *kind* lands in is ``molmcp.host.place``'s table and is proven against +#: every host there, so repeating the matrix here would test that module +#: twice and this one not at all. +HOST = "claude" + +#: A harness catalog declaring one row of every kind that has a host +#: destination, plus one that has none. Bundles are not optional: a catalog is +#: refused outright unless it declares both ``daily`` and ``dev``. +_MANIFEST = """\ +requires = ["harness-catalog"] + +[[component]] +kind = "skill" +name = "daily" +path = "skills/daily/SKILL.md" + +[[component]] +kind = "skill" +name = "review" +path = "skills/review/SKILL.md" + +[[component]] +kind = "agent" +name = "planner" +path = "agents/planner.md" + +[[component]] +kind = "rule" +name = "style" +path = "rules/style.md" + +[[component]] +kind = "provider" +name = "demo" +path = "providers/demo/plane.py" +entrypoint = "plane:build" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.daily", "skill.review", "provider.demo"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["agent.planner", "rule.style"] +""" + +#: A catalog that aims a skill row straight at the managed usage skill. +#: ``install_skill`` owns that directory, and the placement seam refuses it. +_CLOBBER_MANIFEST = """\ +requires = ["harness-catalog"] + +[[component]] +kind = "skill" +name = "molcrafts" +path = "skills/molcrafts/SKILL.md" + +[[component]] +kind = "skill" +name = "review" +path = "skills/review/SKILL.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.molcrafts"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.review"] +""" + +#: A catalog whose components live under a subdirectory of the tree. Used for +#: the second source of the multi-source tests: resolved under the *other* +#: source's root, none of its files exists at all. +_ROOTED_MANIFEST = """\ +requires = ["harness-catalog"] +component_root = "harness" + +[[component]] +kind = "skill" +name = "private-note" +path = "skills/private-note/SKILL.md" + +[[component]] +kind = "bundle" +name = "daily" +members = ["skill.private-note"] + +[[component]] +kind = "bundle" +name = "dev" +members = ["skill.private-note"] +""" + +_DAILY_SKILL = "# daily skill\n" +_REVIEW_SKILL = "# review skill\n" +_PLANNER_AGENT = "# planner agent\n" +_STYLE_RULE = "# style rule\n" +_PROVIDER_MODULE = "def build():\n return None\n" +_PRIVATE_SKILL = "# private note\n" +_CLOBBER_TEXT = "# not the constitution\n" +_SCRATCH = "still being edited\n" + +#: Identity for the commits made here, passed per invocation so no developer's +#: global git config is read and none is written to ``tmp_path``. +_IDENTITY = ( + "-c", + "user.name=molmcp tests", + "-c", + "user.email=tests@molmcp.invalid", +) + + +# -- a real repository, built here ------------------------------------------ +# +# ``tests/test_cli_harness.py`` builds one the same way, and its helpers are +# private names in a module this change does not touch, so they are mirrored +# rather than imported: a test of the read half that breaks when the write +# half's tests are refactored is coupling this suite does not need. + + +def _git(root: Path, *args: str) -> str: + """Run one git command inside *root* and return its stripped stdout.""" + result = subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + text=True, + ) + return result.stdout.strip() + + +def _write(path: Path, text: str) -> None: + """Write *text* to *path*, creating the parent directories it needs.""" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + + +def _commit(root: Path, message: str) -> str: + """Commit everything currently in *root* and return the new SHA.""" + _git(root, "add", "-A") + _git(root, *_IDENTITY, "commit", "--no-gpg-sign", "-q", "-m", message) + return _git(root, "rev-parse", "HEAD") + + +def _harness_checkout(root: Path) -> tuple[Path, str]: + """A one-commit harness checkout of :data:`_MANIFEST`, and its ``HEAD``.""" + root.mkdir(parents=True, exist_ok=True) + _git(root, "init", "-q", "--initial-branch=main") + _write(root / "harness.toml", _MANIFEST) + _write(root / "skills" / "daily" / "SKILL.md", _DAILY_SKILL) + _write(root / "skills" / "review" / "SKILL.md", _REVIEW_SKILL) + _write(root / "agents" / "planner.md", _PLANNER_AGENT) + _write(root / "rules" / "style.md", _STYLE_RULE) + _write(root / "providers" / "demo" / "plane.py", _PROVIDER_MODULE) + return root, _commit(root, "first") + + +def _clobber_checkout(root: Path) -> tuple[Path, str]: + """A checkout whose catalog claims the managed usage skill's own path.""" + root.mkdir(parents=True, exist_ok=True) + _git(root, "init", "-q", "--initial-branch=main") + _write(root / "harness.toml", _CLOBBER_MANIFEST) + _write(root / "skills" / "molcrafts" / "SKILL.md", _CLOBBER_TEXT) + _write(root / "skills" / "review" / "SKILL.md", _REVIEW_SKILL) + return root, _commit(root, "first") + + +def _rooted_checkout(root: Path) -> tuple[Path, str]: + """A checkout whose catalog resolves its components under ``harness/``.""" + root.mkdir(parents=True, exist_ok=True) + _git(root, "init", "-q", "--initial-branch=main") + _write(root / "harness.toml", _ROOTED_MANIFEST) + _write( + root / "harness" / "skills" / "private-note" / "SKILL.md", + _PRIVATE_SKILL, + ) + return root, _commit(root, "first") + + +def _bundle_checkout(root: Path) -> Path: + """A ``--source`` checkout: the daily and dev bundles, no catalog at all. + + This is the route ``molmcp init --source`` has always taken, and it is + deliberately *not* a harness checkout: it has no ``harness.toml``, no + commit and no pointer, because the point of asserting it here is that the + activated-commit route was added beside it rather than on top of it. + """ + _write(root / "daily" / "skills" / "notes" / "NOTE.md", "# notes\n") + _write(root / "dev" / "commands" / "spec.md", "# /mol:spec\n") + return root + + +# -- this install ------------------------------------------------------------ + + +def _install(cache: Path, *harness: dict[str, str]) -> None: + """Write the user settings file this install reads its sources from.""" + st.write_settings_file( + st.user_settings_path(), + {"cacheDir": str(cache), "watch": False, "harness": list(harness)}, + ) + + +def _sync(name: str) -> None: + """Run the real sync verb for one source and require that it succeeded.""" + assert cli.main(["harness", "sync", name]) == 0 + + +def _init(*extra: str) -> int: + """Run ``molmcp init`` for :data:`HOST` with any extra flags appended.""" + return cli.main(["init", HOST, *extra]) + + +def _host_file(home: Path, *parts: str) -> Path: + """One path inside the wired host's configuration directory.""" + return home.joinpath(".claude", *parts) + + +def _snapshot(root: Path) -> dict[str, str]: + """Every regular file under *root* by relative POSIX path, with its text. + + Bytes rather than paths, because idempotence is a claim about content: + a second run that rewrote a destination with different text would leave + the same file list behind. + """ + if not root.is_dir(): + return {} + return { + path.relative_to(root).as_posix(): path.read_text(encoding="utf-8") + for path in sorted(root.rglob("*")) + if path.is_file() + } + + +def _packaged_constitution() -> str: + """The usage ``SKILL.md`` ``install_skill`` copies, read from the package.""" + from molmcp import skill as skill_package + + source = Path(skill_package.__file__).parent / "SKILL.md" + return source.read_text(encoding="utf-8") + + +@pytest.fixture(autouse=True) +def _offline_planes(monkeypatch: pytest.MonkeyPatch) -> None: + """Pin the plane list, so no test here depends on installed science deps. + + ``molmcp init`` renders the MCP JSON from whatever providers this machine + can import, which is a fact about the developer's environment rather than + about the resolver under test. + """ + monkeypatch.setattr( + "molmcp.client_config.default_plane_ids", + lambda: ("molcrafts", "molvis"), + ) + + +@pytest.fixture +def cache(home: Path, monkeypatch: pytest.MonkeyPatch, tmp_path: Path) -> Path: + """A scratch cache root, with the working directory pointed away from it. + + The working directory matters twice over: it is where ``load_settings`` + looks for a project settings file, and it must not be the developer's + checkout, or this suite would read that repository's own configuration. + """ + work = tmp_path / "work" + work.mkdir() + monkeypatch.chdir(work) + return tmp_path / "cache" + + +@pytest.fixture +def synced(cache: Path, tmp_path: Path) -> str: + """One synced local source named ``official``; returns its activated SHA.""" + root, head = _harness_checkout(tmp_path / "official") + _install(cache, {"name": "official", "locator": str(root)}) + _sync("official") + return head + + +class TestInitInstallsWhatTheActivatedCatalogDeclares: + """The happy path: one synced source, one ``molmcp init``, files on disk. + + Nothing between the two commands is faked. The pointer the sync promoted + is the pointer this reads, and the tree it published is the tree these + files are copied out of. + """ + + def test_every_declared_skill_lands_in_the_hosts_skills_directory( + self, home: Path, synced: str + ) -> None: + """Both catalog skills, with the bytes the commit holds. + + ``skills/daily/SKILL.md`` arrives as ``daily/SKILL.md`` under the + host's ``skills/``: the kind prefix is catalog grammar and is stripped + before the row crosses into ``molmcp.host``. + """ + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").read_text( + encoding="utf-8" + ) == (_DAILY_SKILL) + assert _host_file(home, "skills", "review", "SKILL.md").read_text( + encoding="utf-8" + ) == (_REVIEW_SKILL) + + def test_an_agent_row_lands_in_the_hosts_agents_directory( + self, home: Path, synced: str + ) -> None: + assert _init() == 0 + + assert _host_file(home, "agents", "planner.md").read_text(encoding="utf-8") == ( + _PLANNER_AGENT + ) + + def test_a_rule_row_lands_in_the_hosts_rules_directory( + self, home: Path, synced: str + ) -> None: + assert _init() == 0 + + assert _host_file(home, "rules", "style.md").read_text(encoding="utf-8") == ( + _STYLE_RULE + ) + + def test_a_kind_with_no_host_destination_writes_nothing( + self, home: Path, synced: str + ) -> None: + """A ``provider`` is a plane ``molmcp serve`` mounts, not a host file. + + The catalog declares one, so this proves the row was *seen* and + refused rather than never resolved: the two skills beside it landed. + """ + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").is_file() + assert not _host_file(home, "providers").exists() + assert not _host_file(home, "demo").exists() + + def test_nothing_the_catalog_did_not_declare_reaches_the_host( + self, home: Path, cache: Path, tmp_path: Path + ) -> None: + """The published tree is never globbed; the catalog is the inventory. + + The undeclared file is *committed*, so it is genuinely in the + activated tree — the only thing keeping it out of the host is that no + catalog row names it. + """ + root, _ = _harness_checkout(tmp_path / "official") + _write(root / "skills" / "rogue" / "SKILL.md", "# rogue\n") + _commit(root, "second") + _install(cache, {"name": "official", "locator": str(root)}) + _sync("official") + + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").is_file() + assert not _host_file(home, "skills", "rogue").exists() + + +class TestThePlacementReportSaysWhatHappened: + """The resolver hands back the report, not a bare list of paths. + + Two of the decisions a run makes are invisible in a list of destinations — + that a component was refused, and that a destination already existed — so + they are asserted off the report the primitive returns. + """ + + def test_the_report_names_every_destination_that_was_written( + self, home: Path, synced: str + ) -> None: + # Imported inside the test so the rest of this file still reports a + # behavioural failure rather than one collection error while the + # resolver does not exist yet. + from molmcp.harness_install import install_harness_components + + report = install_harness_components(HOST) + + assert set(report.installed) == { + _host_file(home, "skills", "daily", "SKILL.md"), + _host_file(home, "skills", "review", "SKILL.md"), + _host_file(home, "agents", "planner.md"), + _host_file(home, "rules", "style.md"), + } + + def test_the_report_names_the_refused_row_and_its_reason(self, synced: str) -> None: + from molmcp.harness_install import install_harness_components + + report = install_harness_components(HOST) + + assert report.skipped == (("provider.demo", SKIP_NO_HOST_DESTINATION),) + + def test_a_first_run_replaces_nothing(self, synced: str) -> None: + from molmcp.harness_install import install_harness_components + + report = install_harness_components(HOST) + + assert report.replaced == () + + def test_an_install_with_no_synced_source_reports_an_empty_run( + self, cache: Path, tmp_path: Path + ) -> None: + """Nothing configured is the same answer as nothing activated. + + An empty report rather than a raise: an install that has never been + pointed at a harness is the ordinary one, not a broken one. + """ + from molmcp.harness_install import install_harness_components + + _install(cache) + + report = install_harness_components(HOST) + + assert report.installed == () + assert report.replaced == () + assert report.skipped == () + + +class TestASourceThatWasNeverSyncedIsSkipped: + """A configured source is not a synced one, and the difference is silent. + + The operator may have added an entry and not yet run ``molmcp harness + sync``; that is a state the install passes through, not an error it + reports. The unsynced entry is deliberately **first** in the settings + list, so a resolver that stopped at the first pointer it could not read + would install nothing at all. + """ + + def test_an_unsynced_source_is_not_an_error( + self, cache: Path, tmp_path: Path + ) -> None: + never, _ = _harness_checkout(tmp_path / "never") + _install(cache, {"name": "never", "locator": str(never)}) + + assert _init() == 0 + + def test_an_unsynced_source_installs_none_of_its_components( + self, home: Path, cache: Path, tmp_path: Path + ) -> None: + never, _ = _harness_checkout(tmp_path / "never") + _install(cache, {"name": "never", "locator": str(never)}) + + assert _init() == 0 + + assert not _host_file(home, "skills", "daily").exists() + assert not _host_file(home, "agents", "planner.md").exists() + + def test_a_synced_neighbour_still_installs( + self, home: Path, cache: Path, tmp_path: Path + ) -> None: + never, _ = _rooted_checkout(tmp_path / "never") + official, _ = _harness_checkout(tmp_path / "official") + _install( + cache, + {"name": "never", "locator": str(never)}, + {"name": "official", "locator": str(official)}, + ) + _sync("official") + + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").read_text( + encoding="utf-8" + ) == (_DAILY_SKILL) + assert not _host_file(home, "skills", "private-note").exists() + + +class TestEachSourceResolvesUnderItsOwnRoot: + """Two synced sources, two trees, two ``component_root`` answers. + + The second catalog declares ``component_root = "harness"``, so its one + file sits at ``/harness/skills/private-note/SKILL.md``. Resolved + under the first source's tree — or with the first source's root — that + path does not exist, and ``place_components`` refuses the whole run in + its pre-flight pass. So this is not a cosmetic ordering check: getting the + base wrong installs nothing at all. + """ + + @pytest.fixture + def two_sources(self, cache: Path, tmp_path: Path) -> None: + official, _ = _harness_checkout(tmp_path / "official") + private, _ = _rooted_checkout(tmp_path / "private") + _install( + cache, + {"name": "official", "locator": str(official)}, + {"name": "private", "locator": str(private)}, + ) + _sync("official") + _sync("private") + + def test_the_plain_source_installs_from_the_tree_root( + self, home: Path, two_sources: None + ) -> None: + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").read_text( + encoding="utf-8" + ) == (_DAILY_SKILL) + + def test_the_rooted_source_installs_from_its_own_component_root( + self, home: Path, two_sources: None + ) -> None: + assert _init() == 0 + + assert _host_file(home, "skills", "private-note", "SKILL.md").read_text( + encoding="utf-8" + ) == (_PRIVATE_SKILL) + + +class TestTheManagedUsageSkillSurvives: + """``install_skill`` owns the constitution; a catalog cannot take it. + + This is the one destination the placement seam refuses, and it is why the + new step runs *after* ``install_skill`` in ``cli._init``: the seam skips a + destination inside the managed skill directory, which protects a file that + has already been written and nothing else. + """ + + @pytest.fixture + def clobbering(self, cache: Path, tmp_path: Path) -> None: + root, _ = _clobber_checkout(tmp_path / "official") + _install(cache, {"name": "official", "locator": str(root)}) + _sync("official") + + def test_the_constitution_is_the_packaged_file_after_init( + self, home: Path, clobbering: None + ) -> None: + assert _init() == 0 + + installed = _host_file(home, "skills", "molcrafts", "SKILL.md") + text = installed.read_text(encoding="utf-8") + assert "metadata:" not in text + assert "SYMBOL_NOT_FOUND" in text + assert text != _CLOBBER_TEXT + + def test_the_report_names_the_refusal_rather_than_hiding_it( + self, clobbering: None + ) -> None: + from molmcp.harness_install import install_harness_components + + report = install_harness_components(HOST) + + assert report.skipped == (("skill.molcrafts", SKIP_MANAGED_USAGE_SKILL),) + + def test_the_other_rows_of_that_catalog_still_install( + self, home: Path, clobbering: None + ) -> None: + """The refusal is one row, not the run.""" + assert _init() == 0 + + assert _host_file(home, "skills", "review", "SKILL.md").read_text( + encoding="utf-8" + ) == (_REVIEW_SKILL) + + +class TestOnlyTheActivatedCommitReachesTheHost: + """The working tree of the checkout is not what gets installed. + + Identity is the SHA the pointer names, so what lands is what was + committed at the moment of the sync — an edit made afterwards belongs to + no published commit and reaches nothing until the operator syncs again. + """ + + def test_an_edit_made_after_the_sync_does_not_reach_the_host( + self, home: Path, cache: Path, tmp_path: Path + ) -> None: + root, _ = _harness_checkout(tmp_path / "official") + _install(cache, {"name": "official", "locator": str(root)}) + _sync("official") + _write(root / "skills" / "daily" / "SKILL.md", _SCRATCH) + + assert _init() == 0 + + assert _host_file(home, "skills", "daily", "SKILL.md").read_text( + encoding="utf-8" + ) == (_DAILY_SKILL) + + def test_a_component_declared_but_never_committed_reaches_nothing( + self, home: Path, cache: Path, tmp_path: Path + ) -> None: + """A new row *and* its file, both left uncommitted after the sync. + + The activated commit's catalog has no such row, so the file is not a + component of anything this install serves. + """ + root, _ = _harness_checkout(tmp_path / "official") + _install(cache, {"name": "official", "locator": str(root)}) + _sync("official") + _write( + root / "harness.toml", + _MANIFEST + '\n[[component]]\nkind = "rule"\nname = "draft"\n' + 'path = "rules/draft.md"\n', + ) + _write(root / "rules" / "draft.md", _SCRATCH) + + assert _init() == 0 + + assert _host_file(home, "rules", "style.md").is_file() + assert not _host_file(home, "rules", "draft.md").exists() + + +class TestInstallingTwiceIsIdempotent: + """A second ``molmcp init`` is a no-diff run over the same commit.""" + + def test_the_second_run_leaves_the_same_files_with_the_same_bytes( + self, home: Path, synced: str + ) -> None: + assert _init() == 0 + first = _snapshot(home / ".claude") + + assert _init() == 0 + + assert _snapshot(home / ".claude") == first + + def test_the_second_run_reports_every_destination_as_replaced( + self, synced: str + ) -> None: + """``replaced`` is the whole of ``installed`` on a repeat of one set.""" + from molmcp.harness_install import install_harness_components + + first = install_harness_components(HOST) + second = install_harness_components(HOST) + + assert first.replaced == () + assert second.replaced == second.installed + assert second.installed == first.installed + + +def _imported_targets(path: Path) -> tuple[str, ...]: + """Absolute dotted targets *path* imports, relative imports resolved. + + The same walk ``tests/test_host/test_place.py`` uses, for the reason + ``notes.md:isolation-check-imports`` gives: a substring grep over the + source is both too wide — it hits docstrings, and the docstring of a + module that exists *to stay off* a dependency will name it — and too + narrow, since it cannot see a name assembled by concatenation. Dependency + claims are answered from the import nodes themselves. + + ``from . import harness`` names its target in an alias rather than in + ``node.module``, so aliases are resolved too. That also yields + ``molmcp.components.Activation`` for a symbol import, which is not a + module — harmless here, because every claim made against this walk is + about a dotted *prefix* that no symbol of an allowed module can spell. + + Args: + path: A module file under ``src/molmcp``. + + Returns: + Every dotted target the module imports, in source order. + """ + package = ".".join(("molmcp", *path.relative_to(SRC).parent.parts)) + parts = package.split(".") + found: list[str] = [] + for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))): + if isinstance(node, ast.Import): + found.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + base = parts[: len(parts) - (node.level - 1)] + tail = node.module.split(".") if node.module else [] + module = ".".join([*base, *tail]) + else: + module = node.module or "" + found.append(module) + found.extend(f"{module}.{alias.name}" for alias in node.names) + return tuple(found) + + +def _reaches(targets: tuple[str, ...], dotted: str) -> bool: + """Whether any target is *dotted* itself or a module beneath it. + + Compared segment-wise rather than by ``str.startswith`` alone, so + ``molmcp.harness_install`` is not read as a module inside + ``molmcp.harness``. + """ + return any( + target == dotted or target.startswith(f"{dotted}.") for target in targets + ) + + +class TestTheResolverStaysOffTheWorkerStack: + """``molmcp init`` mounts no plane and must not import one. + + ``molmcp.harness`` carries a module-level + ``from .provider_worker.worker import WorkerProvider``, so importing it + pulls the whole FastMCP-bearing worker stack into the process. The + resolver needs three names — ``Activation``, ``ImmutableGitStore`` and + ``load_harness_catalog`` — and every one of them lives in the stdlib-only + ``molmcp.components`` leaf, so it reaches that leaf directly instead of + inheriting the serve-side reader's cost. + """ + + def test_the_resolver_module_exists_where_this_file_mirrors_it(self) -> None: + assert RESOLVER_SOURCE.is_file() + + @pytest.mark.parametrize("dotted", FORBIDDEN_IMPORTS) + def test_it_imports_nothing_from_the_heavy_side(self, dotted: str) -> None: + assert not _reaches(_imported_targets(RESOLVER_SOURCE), dotted) + + def test_it_reaches_the_stdlib_component_leaf_directly(self) -> None: + assert _reaches(_imported_targets(RESOLVER_SOURCE), REQUIRED_IMPORT) + + def test_it_performs_no_import_the_walk_above_cannot_see(self) -> None: + """No ``importlib.import_module`` to route around the AST check. + + The note that makes this an AST test rather than a grep also names + the one hole an AST walk has, so it is closed here rather than left + to a substring scan of the whole file. + """ + tree = ast.parse(RESOLVER_SOURCE.read_text(encoding="utf-8")) + + dynamic = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Call) + and ( + (isinstance(node.func, ast.Name) and node.func.id == "import_module") + or ( + isinstance(node.func, ast.Attribute) + and node.func.attr in {"import_module", "__import__"} + ) + ) + ] + + assert dynamic == [] diff --git a/tests/test_host/__init__.py b/tests/test_host/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_host/test_install.py b/tests/test_host/test_install.py new file mode 100644 index 0000000..f892fca --- /dev/null +++ b/tests/test_host/test_install.py @@ -0,0 +1,193 @@ +"""Host write primitives: usage skill, daily bundle, adapter pointer, dev tree. + +Every test drives one function of ``molmcp.host.install`` against a fake +checkout under ``tmp_path``. ``Path.home`` is patched to ``tmp_path`` so the +destination tree is the real layout without touching the developer's home. +No environment variable is ever set: the checkout is caller-supplied. +""" + +from __future__ import annotations + +import inspect +import re +from pathlib import Path +from typing import Literal + +import pytest + +import molmcp.skill +from molmcp.host.install import ( + ADAPTER_TEXT, + EXTRA_SKILLS, + install_extra_skills, + install_skill, + write_adapter, +) + +HostName = Literal["grok", "claude", "cursor", "codex"] + +#: Every host this spec wires. One adapter body serves all of them. +ALL_HOSTS: tuple[HostName, ...] = ("grok", "claude", "cursor", "codex") + +#: Fixture markers. A daily body must never appear in a dev destination and +#: a dev body must never appear in the daily ``skills/`` tree. +DAILY_BODY = "DAILY-SKILL-BODY" +DEV_BODY = "DEV-SKILL-BODY" + +#: The managed usage constitution, so an overwrite by another primitive shows. +MANAGED_BODY = "MANAGED-BY-INSTALL-SKILL" + +INSTALL_SOURCE = ( + Path(__file__).resolve().parents[2] / "src" / "molmcp" / "host" / "install.py" +) + +#: The packaged usage constitution ``install_skill`` copies, named the way the +#: production lookup names it: the file beside ``molmcp/skill/__init__.py``. +PACKAGED_SKILL = Path(molmcp.skill.__file__).resolve().parent / "SKILL.md" + + +@pytest.fixture +def home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point ``Path.home()`` at ``tmp_path`` — never at a real home.""" + monkeypatch.setattr(Path, "home", lambda: tmp_path) + return tmp_path + + +@pytest.fixture +def checkout(tmp_path: Path) -> Path: + """A caller-supplied bundle checkout: one daily skill, one dev command.""" + source = tmp_path / "checkout" + + daily_skill = source / "daily" / "skills" / "daily-demo" / "SKILL.md" + daily_skill.parent.mkdir(parents=True) + daily_skill.write_text(f"# daily-demo\n\n{DAILY_BODY}\n", encoding="utf-8") + + dev_command = source / "dev" / "commands" / "spec.md" + dev_command.parent.mkdir(parents=True) + dev_command.write_text(f"# spec\n\n{DEV_BODY}\n", encoding="utf-8") + + dev_skill = source / "dev" / "skills" / "spec" / "SKILL.md" + dev_skill.parent.mkdir(parents=True) + dev_skill.write_text(f"# spec\n\n{DEV_BODY}\n", encoding="utf-8") + + return source + + +def _file_bodies(root: Path) -> list[str]: + """Text of every regular file under *root*; empty when *root* is absent.""" + if not root.is_dir(): + return [] + return [ + path.read_text(encoding="utf-8") + for path in sorted(root.rglob("*")) + if path.is_file() + ] + + +class TestInstallSkill: + """Writes the usage constitution and nothing else.""" + + def test_the_written_file_is_remapped_for_the_host(self, home: Path) -> None: + install_skill("grok") + + skill = home / ".grok" / "skills" / "molcrafts" / "SKILL.md" + text = skill.read_text(encoding="utf-8") + assert "when-to-use:" in text + assert "metadata:" not in text + assert skill != PACKAGED_SKILL + + def test_the_template_carries_the_packaged_marker(self, home: Path) -> None: + install_skill("grok") + + skill = home / ".grok" / "skills" / "molcrafts" / "SKILL.md" + assert "SYMBOL_NOT_FOUND" in skill.read_text(encoding="utf-8") + + def test_it_returns_the_path_it_wrote(self, home: Path) -> None: + written = install_skill("grok") + + assert written == home / ".grok" / "skills" / "molcrafts" / "SKILL.md" + + def test_it_does_not_write_the_adapter(self, home: Path) -> None: + install_skill("grok") + + assert not (home / ".grok" / "molmcp-adapter.md").exists() + + def test_it_creates_no_other_skill_directory(self, home: Path) -> None: + install_skill("grok") + + skills = home / ".grok" / "skills" + assert sorted(path.name for path in skills.iterdir()) == ["molcrafts"] + + def test_its_signature_takes_no_source_argument(self) -> None: + assert list(inspect.signature(install_skill).parameters) == ["host"] + + +class TestInstallExtraSkills: + """Writes packaged extras beside the constitution, never into it.""" + + def test_the_catalog_is_the_frozen_tuple(self) -> None: + assert EXTRA_SKILLS == ("molexp-plan",) + + def test_it_writes_molexp_plan_remapped_for_the_host(self, home: Path) -> None: + written = install_extra_skills("grok") + + dest = home / ".grok" / "skills" / "molexp-plan" / "SKILL.md" + assert written == (dest,) + text = dest.read_text(encoding="utf-8") + assert "name: molexp-plan" in text + assert "One step per turn" in text + assert "when-to-use:" in text + assert "metadata:" not in text + + def test_it_does_not_write_the_constitution(self, home: Path) -> None: + install_extra_skills("grok") + + skills = home / ".grok" / "skills" + assert sorted(path.name for path in skills.iterdir()) == ["molexp-plan"] + + def test_its_signature_takes_no_source_argument(self) -> None: + assert list(inspect.signature(install_extra_skills).parameters) == ["host"] + + +class TestWriteAdapter: + """A stable pointer file — byte-identical everywhere, forever.""" + + def test_the_written_bytes_are_the_constant(self, home: Path) -> None: + write_adapter("grok") + + adapter = home / ".grok" / "molmcp-adapter.md" + assert adapter.read_bytes() == ADAPTER_TEXT.encode("utf-8") + + def test_it_returns_the_path_it_wrote(self, home: Path) -> None: + written = write_adapter("grok") + + assert written == home / ".grok" / "molmcp-adapter.md" + + def test_the_constant_is_the_pointer_preamble(self) -> None: + assert "# molmcp adapter" in ADAPTER_TEXT + assert "pointer, not a constitution" in ADAPTER_TEXT + + def test_it_carries_no_skill_bodies(self) -> None: + assert DEV_BODY not in ADAPTER_TEXT + assert DAILY_BODY not in ADAPTER_TEXT + + def test_it_carries_no_machine_specific_home_path(self, home: Path) -> None: + written = write_adapter("grok") + + assert str(home) not in written.read_text(encoding="utf-8") + + def test_it_carries_no_timestamp(self) -> None: + assert re.search(r"\d{4}-\d{2}-\d{2}", ADAPTER_TEXT) is None + + def test_it_carries_no_content_hash(self) -> None: + assert re.search(r"\b[0-9a-f]{40,64}\b", ADAPTER_TEXT) is None + + def test_it_takes_no_source_argument(self) -> None: + assert list(inspect.signature(write_adapter).parameters) == ["host"] + + def test_every_host_gets_byte_identical_content(self, home: Path) -> None: + bodies = { + host: write_adapter(host).read_text(encoding="utf-8") for host in ALL_HOSTS + } + + assert set(bodies.values()) == {ADAPTER_TEXT} diff --git a/tests/test_host/test_layout.py b/tests/test_host/test_layout.py new file mode 100644 index 0000000..e1025d1 --- /dev/null +++ b/tests/test_host/test_layout.py @@ -0,0 +1,295 @@ +"""HostLayout: the single host path table and its thin path readers. + +Mirrors ``src/molmcp/host/layout.py`` for spec +``autonomous-harness-evolution-07-host-adapter`` (ac-001, ac-002, ac-003). +Every tuple below is hard-coded from the spec's HostLayout field table, which +is authoritative; ``mcp_json`` and ``skill_dir`` additionally repeat today's +``client_config._HOST_PATHS`` / ``_HOST_SKILL_DIRS`` destinations, because +moving the table must not move where ``molmcp init`` writes. + +Home is redirected by patching ``pathlib.Path.home``; no environment variable +selects a destination here or in production. +""" + +from __future__ import annotations + +import ast +import dataclasses +import pathlib + +import pytest + +from molmcp.host.layout import ( + HOSTS, + SKILL_NAME, + Host, + HostLayout, + default_skill_dir, + default_write_path, + layout_for, +) + +#: Insertion order of ``HOSTS``; ``cli.py`` derives its ``--help`` choices +#: from it, so the order is part of the contract. +HOST_ORDER: tuple[Host, ...] = ("grok", "claude", "cursor", "codex") + +#: The spec's HostLayout field table, verbatim. Every value is a path tuple +#: relative to ``Path.home()``. +LAYOUTS: dict[Host, dict[str, tuple[str, ...]]] = { + "grok": { + "mcp_json": (".mcp.json",), + "skill_dir": (".grok", "skills", "molcrafts"), + "adapter": (".grok", "molmcp-adapter.md"), + "agents": (".grok", "agents"), + "rules": (".grok", "rules"), + }, + "claude": { + "mcp_json": (".claude.json",), + "skill_dir": (".claude", "skills", "molcrafts"), + "adapter": (".claude", "molmcp-adapter.md"), + "agents": (".claude", "agents"), + "rules": (".claude", "rules"), + }, + "cursor": { + "mcp_json": (".cursor", "mcp.json"), + "skill_dir": (".cursor", "skills", "molcrafts"), + "adapter": (".cursor", "molmcp-adapter.md"), + "agents": (".cursor", "agents"), + "rules": (".cursor", "rules"), + }, + "codex": { + "mcp_json": (".codex", "mcp.json"), + "skill_dir": (".codex", "skills", "molcrafts"), + "adapter": (".codex", "molmcp-adapter.md"), + "agents": (".codex", "agents"), + "rules": (".codex", "rules"), + }, +} + +#: Fields carried by every record, per the spec table. +FIELD_NAMES = frozenset( + { + "mcp_json", + "skill_dir", + "adapter", + "agents", + "rules", + } +) + +#: Catalog-placed destinations (usage skill and MCP JSON stay elsewhere). +BUNDLE_FIELDS = ("adapter", "agents", "rules") + +SRC = pathlib.Path(__file__).resolve().parents[2] / "src" / "molmcp" +HOST_PKG = SRC / "host" + +#: ac-002: ``host/`` is Layer 2 and never imports an outer layer. +FORBIDDEN_ROOTS: tuple[str, ...] = ( + "molmcp.client_config", + "molmcp.cli", + "molmcp.server", + "molmcp.providers", + "molmcp.discovery", + "molmcp.components", + "molmcp.harness", +) + + +def _imported_names(path: pathlib.Path) -> tuple[str, ...]: + """Absolute dotted targets imported by *path*, relative imports resolved.""" + package = ".".join(("molmcp", *path.relative_to(SRC).parent.parts)) + parts = package.split(".") + found: list[str] = [] + for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))): + if isinstance(node, ast.Import): + found.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + base = parts[: len(parts) - (node.level - 1)] + tail = node.module.split(".") if node.module else [] + prefix = ".".join([*base, *tail]) + else: + prefix = node.module or "" + found.append(prefix) + found.extend(f"{prefix}.{alias.name}" for alias in node.names) + return tuple(found) + + +def _offends(dotted: str) -> bool: + return any( + dotted == root or dotted.startswith(f"{root}.") for root in FORBIDDEN_ROOTS + ) + + +class TestHostLayout: + """``layout_for`` / ``default_write_path`` / ``default_skill_dir``.""" + + # --- Basics ------------------------------------------------------- + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_layout_for_returns_todays_mcp_json(self, host: Host) -> None: + assert layout_for(host).mcp_json == LAYOUTS[host]["mcp_json"] + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_layout_for_returns_todays_skill_dir(self, host: Host) -> None: + assert layout_for(host).skill_dir == LAYOUTS[host]["skill_dir"] + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_layout_for_returns_the_bundle_destinations(self, host: Host) -> None: + layout = layout_for(host) + + actual = {name: getattr(layout, name) for name in BUNDLE_FIELDS} + + assert actual == {name: LAYOUTS[host][name] for name in BUNDLE_FIELDS} + + def test_the_record_declares_exactly_the_spec_table_fields(self) -> None: + names = {field.name for field in dataclasses.fields(HostLayout)} + + assert names == FIELD_NAMES + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_skill_dir_ends_with_the_managed_skill_name(self, host: Host) -> None: + assert SKILL_NAME == "molcrafts" + assert layout_for(host).skill_dir[-1] == SKILL_NAME + + def test_hosts_holds_exactly_the_four_known_hosts(self) -> None: + assert set(HOSTS) == {"grok", "claude", "cursor", "codex"} + + def test_hosts_iterates_in_cli_choice_order(self) -> None: + assert tuple(HOSTS) == HOST_ORDER + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_default_write_path_joins_home_with_mcp_json( + self, host: Host, tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + monkeypatch.setattr(pathlib.Path, "home", lambda: tmp_path) + + assert default_write_path(host) == tmp_path.joinpath(*LAYOUTS[host]["mcp_json"]) + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_default_skill_dir_joins_home_with_skill_dir( + self, host: Host, tmp_path: pathlib.Path, monkeypatch: pytest.MonkeyPatch + ) -> None: + monkeypatch.setattr(pathlib.Path, "home", lambda: tmp_path) + + assert default_skill_dir(host) == tmp_path.joinpath(*LAYOUTS[host]["skill_dir"]) + + # --- Immutability ------------------------------------------------- + + def test_the_record_is_frozen(self) -> None: + layout = layout_for("grok") + + with pytest.raises(dataclasses.FrozenInstanceError): + layout.mcp_json = (".other.json",) + + def test_the_record_uses_slots(self) -> None: + layout = layout_for("grok") + + assert "__slots__" in vars(HostLayout) + assert not hasattr(layout, "__dict__") + + @pytest.mark.parametrize("host", HOST_ORDER) + def test_every_field_value_is_a_tuple_of_str(self, host: Host) -> None: + layout = layout_for(host) + + for field in dataclasses.fields(HostLayout): + value = getattr(layout, field.name) + assert isinstance(value, tuple), field.name + assert all(isinstance(part, str) for part in value), field.name + + # --- Edge --------------------------------------------------------- + + def test_layout_for_rejects_an_unknown_host(self) -> None: + with pytest.raises(ValueError, match="emacs"): + layout_for("emacs") + + def test_default_write_path_rejects_an_unknown_host(self) -> None: + with pytest.raises(ValueError, match="emacs"): + default_write_path("emacs") + + def test_default_skill_dir_rejects_an_unknown_host(self) -> None: + with pytest.raises(ValueError, match="emacs"): + default_skill_dir("emacs") + + # --- Layering (ac-002) -------------------------------------------- + + def test_no_host_module_imports_an_outer_layer(self) -> None: + assert HOST_PKG.is_dir(), f"{HOST_PKG} does not exist" + modules = sorted(HOST_PKG.rglob("*.py")) + assert modules, f"no modules under {HOST_PKG}" + + offenders = { + module.relative_to(SRC).as_posix(): [ + dotted for dotted in _imported_names(module) if _offends(dotted) + ] + for module in modules + } + + assert {name: hits for name, hits in offenders.items() if hits} == {} + + +_FENCE = """\ +--- +name: daily +description: > + A daily skill +when-to-use: every morning +user-invocable: false +disable-model-invocation: true +argument-hint: "" +tools: Read, Grep +model: sonnet +metadata: + author: molmcp +--- +# body +""" + + +class TestRemapFrontmatter: + def test_grok_keeps_when_to_use_and_drops_tools(self) -> None: + from molmcp.host.layout import remap_frontmatter + + out = remap_frontmatter(_FENCE, "grok") + assert "when-to-use: every morning" in out + assert 'argument-hint: ""' in out + assert "tools:" not in out + assert "metadata:" not in out + assert "# body" in out + + def test_claude_drops_when_to_use(self) -> None: + from molmcp.host.layout import remap_frontmatter + + out = remap_frontmatter(_FENCE, "claude") + assert "when-to-use:" not in out + assert "user-invocable: false" in out + assert "argument-hint:" in out + + def test_cursor_keeps_only_three_keys(self) -> None: + from molmcp.host.layout import remap_frontmatter + + out = remap_frontmatter(_FENCE, "cursor") + assert "name: daily" in out + assert "disable-model-invocation: true" in out + assert "user-invocable:" not in out + assert "argument-hint:" not in out + + def test_codex_keeps_name_and_description(self) -> None: + from molmcp.host.layout import remap_frontmatter + + out = remap_frontmatter(_FENCE, "codex") + assert "name: daily" in out + assert "description: >" in out + assert "disable-model-invocation:" not in out + + def test_no_fence_is_unchanged(self) -> None: + from molmcp.host.layout import remap_frontmatter + + raw = "# just a rule\n" + assert remap_frontmatter(raw, "grok") == raw + + def test_folded_description_continuations_stay(self) -> None: + from molmcp.host.layout import remap_frontmatter + + out = remap_frontmatter(_FENCE, "grok") + assert " A daily skill" in out diff --git a/tests/test_host/test_place.py b/tests/test_host/test_place.py new file mode 100644 index 0000000..3b9565c --- /dev/null +++ b/tests/test_host/test_place.py @@ -0,0 +1,665 @@ +"""Placing catalog-declared components from an activated commit into a host. + +Mirrors ``src/molmcp/host/place.py``. This is the last link of the pinned +chain: ``molmcp harness sync`` publishes a commit tree under +``cacheDir/harness/commits//tree`` and moves that source's activation +pointer onto it, and ``molmcp init`` has to be able to install what that +tree's ``harness.toml`` declares. It could not: ``install.materialize_daily`` +reads ``/daily/skills//``, a layout a harness checkout does not +have, so against a real one it installs nothing. + +The seam this module tests is the fix, and its shape is the design decision +under test. ``src/molmcp/host/`` is stdlib-only and imports no other +``molmcp`` module; that convention is kept, so ``host/`` is never told what a +``HarnessCatalog`` is. Instead the caller — which already holds the fold, the +checkout trees and each catalog's ``component_root`` — resolves every +component down to a plain description of one file to place, and hands those +descriptions over: + + ComponentFile(id=..., kind=..., relative=..., source=...) + +Four stdlib-expressible fields, no catalog type among them. ``host/`` then +owns exactly one thing the caller does not: which host directory a *kind* +belongs in. ``place_components`` answers with a ``PlacementReport`` rather +than a bare tuple, because two of the rules below are about what a run +*decided* — that a component was skipped, and that a destination already +existed — and neither is visible in a list of paths. + +Two rules are load-bearing enough to say out loud here: + +* **The tree is never globbed.** ``place_components`` copies the files it is + handed and reads nothing else, which is how "what is installed came from + the activated commit" survives at this seam. The commit-pinning half of + that chain — that the published tree holds committed content only — is + proven where the publishing happens, not here. +* **The managed usage skill is never clobbered.** ``install_skill`` owns + ``skills/molcrafts/``; ``materialize_daily`` already refuses to write into + a directory named ``SKILL_NAME``, and that protection has to survive a + catalog that declares a component there. + +``Path.home`` is patched to ``tmp_path`` so every destination is the real +layout without touching the developer's home. No environment variable is +read: ``tests/test_no_env_switches.py`` already scans every module under +``src/molmcp`` for that, so it is not restated here. +""" + +from __future__ import annotations + +import ast +import dataclasses +import inspect +import sys +from pathlib import Path +from typing import Literal + +import pytest + +from molmcp.host import ( + SKILL_NAME, + SKIP_MANAGED_USAGE_SKILL, + SKIP_NO_HOST_DESTINATION, + ComponentFile, + PlacementReport, + place_components, +) + +HostName = Literal["grok", "claude", "cursor", "codex"] + +SRC = Path(__file__).resolve().parents[2] / "src" / "molmcp" + +#: The module under test, read as data by the isolation check below. +PLACE_SOURCE = SRC / "host" / "place.py" + +#: Fixture markers. Each one names where its file came from, so a body that +#: turns up in the wrong destination says so. +SKILL_BODY = "CATALOG-SKILL-BODY" +AGENT_BODY = "CATALOG-AGENT-BODY" +RULE_BODY = "CATALOG-RULE-BODY" +PROVIDER_BODY = "CATALOG-PROVIDER-BODY" +OVERLAY_BODY = "CATALOG-OVERLAY-BODY" + +#: A file that exists in the developer's working checkout but was never in +#: the commit the pointer names. Nothing carrying this may reach a host. +UNCOMMITTED_BODY = "UNCOMMITTED-WORKING-TREE-BODY" + +#: A file sitting beside a declared component inside the published tree that +#: no catalog row mentions. The tree is an inventory, not a directory to walk. +UNDECLARED_BODY = "UNDECLARED-SIBLING-BODY" + +#: The managed usage constitution ``install_skill`` writes, so an overwrite +#: by this module would show as a changed body. +MANAGED_BODY = "MANAGED-BY-INSTALL-SKILL" + + +@pytest.fixture +def home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point ``Path.home()`` at ``tmp_path`` — never at a real home.""" + monkeypatch.setattr(Path, "home", lambda: tmp_path) + return tmp_path + + +@pytest.fixture +def tree(tmp_path: Path) -> Path: + """One activated commit tree, laid out as ``harness sync`` publishes it. + + ``commits//tree/`` with a ``component_root`` of ``harness``, holding + one file per component kind plus one undeclared sibling of the skill. + """ + sha = "9f1c3b2a7d4e0165c8a9b3d27e5f10486c73ab92" + root = tmp_path / "cache" / "harness" / "commits" / sha / "tree" / "harness" + + files = { + "skills/daily/SKILL.md": SKILL_BODY, + "skills/daily/NOTES.md": UNDECLARED_BODY, + "agents/librarian/AGENT.md": AGENT_BODY, + "rules/no-invented-api.md": RULE_BODY, + "providers/bench/provider.py": PROVIDER_BODY, + "overlays/molpy/overlay.py": OVERLAY_BODY, + } + for relative, body in files.items(): + path = root / relative + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(f"# {relative}\n\n{body}\n", encoding="utf-8") + return root + + +def _skill(tree: Path) -> ComponentFile: + """The ``skill.daily`` row of the example catalog, already resolved.""" + return ComponentFile( + id="skill.daily", + kind="skill", + relative="daily/SKILL.md", + source=tree / "skills" / "daily" / "SKILL.md", + ) + + +def _agent(tree: Path) -> ComponentFile: + """The ``agent.librarian`` row, already resolved.""" + return ComponentFile( + id="agent.librarian", + kind="agent", + relative="librarian/AGENT.md", + source=tree / "agents" / "librarian" / "AGENT.md", + ) + + +def _rule(tree: Path) -> ComponentFile: + """The ``rule.no-invented-api`` row, already resolved.""" + return ComponentFile( + id="rule.no-invented-api", + kind="rule", + relative="no-invented-api.md", + source=tree / "rules" / "no-invented-api.md", + ) + + +def _provider(tree: Path) -> ComponentFile: + """The ``provider.bench`` row — a plane, not a file a host installs.""" + return ComponentFile( + id="provider.bench", + kind="provider", + relative="bench/provider.py", + source=tree / "providers" / "bench" / "provider.py", + ) + + +def _overlay(tree: Path) -> ComponentFile: + """The ``overlay.molpy`` row — knowledge for discovery, not for a host.""" + return ComponentFile( + id="overlay.molpy", + kind="overlay", + relative="molpy/overlay.py", + source=tree / "overlays" / "molpy" / "overlay.py", + ) + + +def _bodies(root: Path) -> list[str]: + """Text of every regular file under *root*; empty when *root* is absent.""" + if not root.is_dir(): + return [] + return [ + path.read_text(encoding="utf-8") + for path in sorted(root.rglob("*")) + if path.is_file() + ] + + +def _file_set(root: Path) -> set[Path]: + """Every regular file under *root*, relative to it.""" + if not root.is_dir(): + return set() + return {path.relative_to(root) for path in root.rglob("*") if path.is_file()} + + +def _imported_names(path: Path) -> tuple[str, ...]: + """Absolute dotted targets imported by *path*, relative imports resolved. + + The same walk ``test_layout.py`` uses: a substring grep over the source + is both too wide (it hits docstrings) and too narrow (it misses a name + built by concatenation), so dependency claims are answered from the + import nodes themselves. + """ + package = ".".join(("molmcp", *path.relative_to(SRC).parent.parts)) + parts = package.split(".") + found: list[str] = [] + for node in ast.walk(ast.parse(path.read_text(encoding="utf-8"))): + if isinstance(node, ast.Import): + found.extend(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom): + if node.level: + base = parts[: len(parts) - (node.level - 1)] + tail = node.module.split(".") if node.module else [] + found.append(".".join([*base, *tail])) + else: + found.append(node.module or "") + return tuple(found) + + +class TestComponentFile: + """The seam's input: one file to place, described in stdlib types only.""" + + # --- Basics ------------------------------------------------------- + + def test_it_carries_the_four_fields_the_seam_needs(self) -> None: + names = {field.name for field in dataclasses.fields(ComponentFile)} + + assert names == {"id", "kind", "relative", "source"} + + def test_the_id_is_the_catalog_id_unchanged(self, tree: Path) -> None: + assert _skill(tree).id == "skill.daily" + + def test_the_kind_is_the_catalog_kind_as_a_plain_string(self, tree: Path) -> None: + """A ``str``, not a ``ComponentKind``, and not validated here. + + The carrier stays dumb: which kinds have a host destination is + ``place_components``' table and has exactly one owner. That is why + an unknown kind is refused there rather than at construction. + """ + kind = _skill(tree).kind + + assert kind == "skill" + assert type(kind) is str + + def test_the_relative_path_is_stripped_of_the_catalog_prefix( + self, tree: Path + ) -> None: + """``skills/daily/SKILL.md`` arrives as ``daily/SKILL.md``. + + The caller strips ``KIND_PATH_PREFIX``; that prefix is catalog + grammar and ``host/`` never learns it. + """ + assert _skill(tree).relative == "daily/SKILL.md" + + def test_the_source_is_an_absolute_path_in_the_activated_tree( + self, tree: Path + ) -> None: + """The caller has already joined ``component_root`` onto the tree. + + One base per row, because a fold can hold several sources and each + one resolves under its own ``ComponentFold.root_for`` answer. + """ + source = _skill(tree).source + + assert source.is_absolute() + assert source.read_text(encoding="utf-8").count(SKILL_BODY) == 1 + + # --- Immutability ------------------------------------------------- + + def test_it_is_frozen(self, tree: Path) -> None: + component = _skill(tree) + + with pytest.raises(dataclasses.FrozenInstanceError): + component.kind = "agent" # type: ignore[misc] + + +class TestPlacementReport: + """The seam's output: what the run placed, replaced, and refused.""" + + # --- Basics ------------------------------------------------------- + + def test_it_carries_the_three_fields_a_run_decides(self) -> None: + names = {field.name for field in dataclasses.fields(PlacementReport)} + + assert names == {"installed", "replaced", "skipped"} + + def test_the_two_skip_reasons_are_distinct_strings(self) -> None: + """A reader must be able to tell the two refusals apart.""" + assert SKIP_NO_HOST_DESTINATION != SKIP_MANAGED_USAGE_SKILL + assert SKIP_NO_HOST_DESTINATION and SKIP_MANAGED_USAGE_SKILL + + # --- Immutability ------------------------------------------------- + + def test_it_is_frozen(self, home: Path) -> None: + report = place_components("grok", ()) + + with pytest.raises(dataclasses.FrozenInstanceError): + report.installed = () # type: ignore[misc] + + +class TestPlaceComponents: + """The kind decides the destination; the caller decides the files.""" + + # --- Basics ------------------------------------------------------- + + def test_its_signature_takes_a_host_and_the_components(self) -> None: + parameters = list(inspect.signature(place_components).parameters) + + assert parameters == ["host", "components"] + + def test_a_skill_component_lands_in_the_host_skills_tree( + self, home: Path, tree: Path + ) -> None: + place_components("grok", (_skill(tree),)) + + skill = home / ".grok" / "skills" / "daily" / "SKILL.md" + assert SKILL_BODY in skill.read_text(encoding="utf-8") + + def test_every_declared_skill_lands(self, home: Path, tree: Path) -> None: + """A catalog declaring several skills installs all of them.""" + second = tree / "skills" / "review" / "SKILL.md" + second.parent.mkdir(parents=True) + second.write_text(f"# review\n\n{SKILL_BODY}\n", encoding="utf-8") + review = ComponentFile( + id="skill.review", + kind="skill", + relative="review/SKILL.md", + source=second, + ) + + place_components("grok", (_skill(tree), review)) + + skills = home / ".grok" / "skills" + assert _file_set(skills) == { + Path("daily") / "SKILL.md", + Path("review") / "SKILL.md", + } + + def test_an_agent_component_lands_in_the_host_agents_tree( + self, home: Path, tree: Path + ) -> None: + place_components("grok", (_agent(tree),)) + + agent = home / ".grok" / "agents" / "librarian" / "AGENT.md" + assert AGENT_BODY in agent.read_text(encoding="utf-8") + + def test_a_rule_component_lands_in_the_host_rules_tree( + self, home: Path, tree: Path + ) -> None: + place_components("grok", (_rule(tree),)) + + rule = home / ".grok" / "rules" / "no-invented-api.md" + assert RULE_BODY in rule.read_text(encoding="utf-8") + + def test_an_agent_is_not_installed_as_a_skill(self, home: Path, tree: Path) -> None: + place_components("grok", (_agent(tree),)) + + assert AGENT_BODY not in "".join(_bodies(home / ".grok" / "skills")) + + def test_a_provider_component_is_not_installed_as_a_skill( + self, home: Path, tree: Path + ) -> None: + """A provider is a plane this process mounts, not a host file.""" + place_components("grok", (_provider(tree),)) + + assert PROVIDER_BODY not in "".join(_bodies(home / ".grok")) + + def test_an_overlay_component_is_not_installed_anywhere( + self, home: Path, tree: Path + ) -> None: + place_components("grok", (_overlay(tree),)) + + assert OVERLAY_BODY not in "".join(_bodies(home / ".grok")) + + def test_a_provider_is_reported_skipped_with_a_reason( + self, home: Path, tree: Path + ) -> None: + report = place_components("grok", (_provider(tree),)) + + assert report.installed == () + assert report.skipped == (("provider.bench", SKIP_NO_HOST_DESTINATION),) + + def test_a_mixed_bundle_installs_three_kinds_and_skips_two( + self, home: Path, tree: Path + ) -> None: + report = place_components( + "grok", + ( + _skill(tree), + _agent(tree), + _rule(tree), + _provider(tree), + _overlay(tree), + ), + ) + + assert report.installed == ( + home / ".grok" / "skills" / "daily" / "SKILL.md", + home / ".grok" / "agents" / "librarian" / "AGENT.md", + home / ".grok" / "rules" / "no-invented-api.md", + ) + assert report.skipped == ( + ("provider.bench", SKIP_NO_HOST_DESTINATION), + ("overlay.molpy", SKIP_NO_HOST_DESTINATION), + ) + + def test_nothing_to_place_writes_nothing(self, home: Path) -> None: + report = place_components("grok", ()) + + assert report == PlacementReport(installed=(), replaced=(), skipped=()) + assert not (home / ".grok").exists() + + @pytest.mark.parametrize("host", ["grok", "claude", "cursor", "codex"]) + def test_every_host_gets_its_own_skills_tree( + self, home: Path, tree: Path, host: HostName + ) -> None: + place_components(host, (_skill(tree),)) + + assert any( + SKILL_BODY in body + for body in _bodies(home / f".{host}" / "skills" / "daily") + ) + + # --- What is installed came from the activated commit -------------- + + def test_an_undeclared_sibling_in_the_tree_is_never_installed( + self, home: Path, tree: Path + ) -> None: + """The tree is an inventory, not a directory to walk. + + ``skills/daily/NOTES.md`` sits beside the declared ``SKILL.md`` and + no catalog row mentions it, so nothing may copy it — this is the + seam's half of "a file nobody declared is not a component". + """ + place_components("grok", (_skill(tree),)) + + assert UNDECLARED_BODY not in "".join(_bodies(home / ".grok")) + + def test_work_left_uncommitted_in_the_checkout_cannot_reach_a_host( + self, home: Path, tmp_path: Path, tree: Path + ) -> None: + """Install after a sync sees the published tree and nothing else. + + The developer's own checkout carries an edit that was never + committed, so it is not in the tree the pointer names. The seam is + handed rows resolved under that tree, and it reads no other + directory — so the edit cannot be installed. That the published + tree holds committed content only is proven where publishing + happens; this is the half that says nothing bypasses it. + """ + working = tmp_path / "checkout" / "harness" + uncommitted = working / "skills" / "daily" / "SKILL.md" + uncommitted.parent.mkdir(parents=True) + uncommitted.write_text(f"# daily\n\n{UNCOMMITTED_BODY}\n", encoding="utf-8") + + place_components("grok", (_skill(tree),)) + + bodies = "".join(_bodies(home / ".grok")) + assert UNCOMMITTED_BODY not in bodies + assert SKILL_BODY in bodies + + # --- Edge --------------------------------------------------------- + + def test_the_managed_usage_skill_is_never_clobbered( + self, home: Path, tree: Path + ) -> None: + """A catalog declaring ``skills/molcrafts/`` does not win that name. + + ``install_skill`` owns the usage constitution; + ``materialize_daily`` already skips a directory named + ``SKILL_NAME`` and that protection has to survive this route. + """ + managed = home / ".grok" / "skills" / SKILL_NAME / "SKILL.md" + managed.parent.mkdir(parents=True) + managed.write_text(MANAGED_BODY, encoding="utf-8") + squatter = tree / "skills" / SKILL_NAME / "SKILL.md" + squatter.parent.mkdir(parents=True) + squatter.write_text(SKILL_BODY, encoding="utf-8") + + place_components( + "grok", + ( + ComponentFile( + id="skill.molcrafts", + kind="skill", + relative=f"{SKILL_NAME}/SKILL.md", + source=squatter, + ), + ), + ) + + assert managed.read_text(encoding="utf-8") == MANAGED_BODY + + def test_the_managed_usage_skill_refusal_is_reported( + self, home: Path, tree: Path + ) -> None: + squatter = tree / "skills" / SKILL_NAME / "SKILL.md" + squatter.parent.mkdir(parents=True) + squatter.write_text(SKILL_BODY, encoding="utf-8") + + report = place_components( + "grok", + ( + ComponentFile( + id="skill.molcrafts", + kind="skill", + relative=f"{SKILL_NAME}/SKILL.md", + source=squatter, + ), + ), + ) + + assert report.installed == () + assert report.skipped == (("skill.molcrafts", SKIP_MANAGED_USAGE_SKILL),) + + def test_a_missing_component_file_names_the_id_and_the_path( + self, home: Path, tree: Path + ) -> None: + missing = tree / "skills" / "ghost" / "SKILL.md" + ghost = ComponentFile( + id="skill.ghost", + kind="skill", + relative="ghost/SKILL.md", + source=missing, + ) + + with pytest.raises(FileNotFoundError) as caught: + place_components("grok", (ghost,)) + + message = str(caught.value) + assert "skill.ghost" in message + assert str(missing) in message + + def test_a_missing_component_file_installs_no_partial_set( + self, home: Path, tree: Path + ) -> None: + """One unresolvable row fails the whole run before anything is written.""" + ghost = ComponentFile( + id="skill.ghost", + kind="skill", + relative="ghost/SKILL.md", + source=tree / "skills" / "ghost" / "SKILL.md", + ) + + with pytest.raises(FileNotFoundError): + place_components("grok", (_skill(tree), ghost)) + + assert _file_set(home / ".grok") == set() + + def test_a_directory_is_not_a_component_file(self, home: Path, tree: Path) -> None: + """A component names one file; a directory fails the same way.""" + directory = ComponentFile( + id="skill.daily", + kind="skill", + relative="daily", + source=tree / "skills" / "daily", + ) + + with pytest.raises(FileNotFoundError, match="skill.daily"): + place_components("grok", (directory,)) + + @pytest.mark.parametrize( + "relative", + ["../evil.md", "daily/../../evil.md", "/etc/evil.md"], + ) + def test_a_relative_path_that_escapes_the_host_root_is_refused( + self, home: Path, tree: Path, relative: str + ) -> None: + escaping = ComponentFile( + id="skill.evil", + kind="skill", + relative=relative, + source=tree / "skills" / "daily" / "SKILL.md", + ) + + with pytest.raises(ValueError, match="skill.evil"): + place_components("grok", (escaping,)) + + def test_an_unknown_kind_is_refused(self, home: Path, tree: Path) -> None: + """Five kinds exist; a sixth means the caller is broken, not the file.""" + unknown = ComponentFile( + id="widget.thing", + kind="widget", + relative="thing.md", + source=tree / "rules" / "no-invented-api.md", + ) + + with pytest.raises(ValueError, match="widget"): + place_components("grok", (unknown,)) + + def test_an_unknown_host_is_refused_before_anything_is_placed( + self, home: Path + ) -> None: + """Host validation first, as in every other primitive of this family.""" + with pytest.raises(ValueError, match="emacs"): + place_components("emacs", ()) + + # --- Lifecycle: a second run replaces, never duplicates ------------- + + def test_the_first_run_reports_nothing_replaced( + self, home: Path, tree: Path + ) -> None: + report = place_components("grok", (_skill(tree), _rule(tree))) + + assert report.replaced == () + assert len(report.installed) == 2 + + def test_a_second_run_writes_the_same_file_set( + self, home: Path, tree: Path + ) -> None: + components = (_skill(tree), _agent(tree), _rule(tree)) + place_components("grok", components) + first = _file_set(home / ".grok") + + place_components("grok", components) + + assert _file_set(home / ".grok") == first + + def test_a_second_run_reports_every_destination_as_replaced( + self, home: Path, tree: Path + ) -> None: + components = (_skill(tree), _rule(tree)) + place_components("grok", components) + + report = place_components("grok", components) + + assert report.replaced == report.installed + assert report.installed != () + + def test_a_changed_source_overwrites_the_destination( + self, home: Path, tree: Path + ) -> None: + place_components("grok", (_skill(tree),)) + (tree / "skills" / "daily" / "SKILL.md").write_text( + f"# daily\n\n{SKILL_BODY}-v2\n", encoding="utf-8" + ) + + place_components("grok", (_skill(tree),)) + + skill = home / ".grok" / "skills" / "daily" / "SKILL.md" + assert f"{SKILL_BODY}-v2" in skill.read_text(encoding="utf-8") + + +class TestPlaceStaysInsideHost: + """``host/`` never learns what a ``HarnessCatalog`` is.""" + + def test_the_module_exists(self) -> None: + assert PLACE_SOURCE.is_file(), f"{PLACE_SOURCE} does not exist" + + def test_it_imports_only_stdlib_and_its_own_package(self) -> None: + """The seam is why this holds, so this is where it is enforced. + + ``test_layout.py`` forbids the five outer layers for the whole + package; this is the stricter rule the injected seam buys — a + component reaches ``host/`` as four plain values, so nothing here + needs ``molmcp.components``, ``molmcp.harness`` or anything else + under ``molmcp`` outside ``molmcp.host``. + """ + offenders = [ + dotted + for dotted in _imported_names(PLACE_SOURCE) + if dotted + and not dotted.startswith("molmcp.host") + and dotted.split(".")[0] not in sys.stdlib_module_names + ] + + assert offenders == [] diff --git a/tests/test_init.py b/tests/test_init.py new file mode 100644 index 0000000..0171067 --- /dev/null +++ b/tests/test_init.py @@ -0,0 +1,232 @@ +"""``molmcp`` re-exports its public names lazily; the façades only forward. + +Importing ``molmcp`` today pulls in ``.server`` and ``.provider``, and both of +those reach ``from fastmcp import FastMCP`` at module scope. A provider running +in a worker subprocess needs ``molmcp.provider_worker.protocol`` and nothing +else: the moment the package body imports FastMCP for it, the child pays for a +server it never builds and the isolation assertion in ``child.py`` can no longer +tell a leak from the import that always happened. + +So both package bodies resolve names through PEP 562 ``__getattr__``. The +public surface (``__all__``) is unchanged — this is a resolution change, not an +API change — and ``__version__`` stays eager because it is metadata, not a +module. +""" + +from __future__ import annotations + +import ast +import importlib.metadata +import os +import subprocess +import sys +from pathlib import Path + +import pytest + +import molmcp + +REPO_ROOT = Path(__file__).resolve().parents[1] +SRC = REPO_ROOT / "src" +PACKAGE_INIT = SRC / "molmcp" / "__init__.py" +WORKER_INIT = SRC / "molmcp" / "provider_worker" / "__init__.py" + +#: Relative submodules that must never be imported by the package body: each +#: one drags FastMCP (directly or transitively) into every ``import molmcp``. +_EAGER_SUBMODULES = frozenset({"mcp_provider", "planes", "provider", "server"}) + +#: The public surface as it stands before the lazy rewrite. Hard-coded so that +#: "resolve it later" can never quietly become "drop it". +_PUBLIC_NAMES = frozenset( + { + "AppConfig", + "CORE_PLANE_ID", + "CollectionIndex", + "ConfigurationError", + "ContextPack", + "MolCraftsContextProvider", + "PROVIDER_ENTRY_POINT_GROUP", + "PlaneInfo", + "PlaneToggle", + "Provider", + "SearchHit", + "SourceBinding", + "__version__", + "create_plane", + "create_server", + "create_stack", + "discover_providers", + "known_plane_ids", + "list_plane_infos", + "load_config", + "provider_available", + "resolve_plane_toggles", + "route_task", + } +) + +#: Imports ``protocol`` the way ``child.py`` will, then reports any FastMCP +#: module that came along for the ride. +_ISOLATION_PROBE = ( + "import molmcp.provider_worker.protocol, sys; " + 'print([m for m in sys.modules if m == "fastmcp" or m.startswith("fastmcp.")])' +) + + +def _parse(path: Path) -> ast.Module: + """Parse ``path``, failing with its name rather than an OSError. + + Args: + path: Source file to parse. + + Returns: + The parsed module. + """ + assert path.is_file(), f"{path} does not exist" + return ast.parse(path.read_text(encoding="utf-8")) + + +def _module_body(path: Path) -> list[ast.stmt]: + """Return the top-level statements of ``path`` — what runs on import.""" + return _parse(path).body + + +def _describe(node: ast.stmt) -> str: + """Name a top-level statement in the vocabulary the façade is allowed.""" + if isinstance(node, ast.Expr) and isinstance(node.value, ast.Constant): + if isinstance(node.value.value, str): + return "docstring" + if isinstance(node, ast.ImportFrom) and node.module == "__future__": + return "future-import" + if isinstance(node, ast.Assign): + targets = [t.id for t in node.targets if isinstance(t, ast.Name)] + if targets == ["__all__"]: + return "__all__" + if isinstance(node, ast.FunctionDef): + return f"def {node.name}" + return f"<{type(node).__name__}>" + + +def _imports_fastmcp(tree: ast.AST) -> bool: + """Report whether any import anywhere in ``tree`` names ``fastmcp``.""" + for node in ast.walk(tree): + if isinstance(node, ast.Import): + if any(a.name.split(".")[0] == "fastmcp" for a in node.names): + return True + if isinstance(node, ast.ImportFrom) and node.module is not None: + if node.module.split(".")[0] == "fastmcp": + return True + return False + + +class TestLazyExports: + """``molmcp/__init__.py`` and ``provider_worker/__init__.py`` as façades.""" + + def test_package_body_imports_no_server_or_provider_submodule(self) -> None: + offenders = sorted( + str(node.module) + for node in _module_body(PACKAGE_INIT) + if isinstance(node, ast.ImportFrom) + and node.level == 1 + and node.module in _EAGER_SUBMODULES + ) + + assert offenders == [], ( + f"molmcp/__init__.py eagerly imports {offenders}; every one of them " + f"reaches FastMCP, so a worker child that only wants " + f"provider_worker.protocol pays for the whole server. Resolve them " + f"in __getattr__ instead." + ) + + def test_public_names_are_unchanged(self) -> None: + assert { + "create_plane", + "create_stack", + "Provider", + "discover_providers", + } <= set(molmcp.__all__) + assert set(molmcp.__all__) == _PUBLIC_NAMES, ( + "moving to lazy resolution must not add or drop a public name" + ) + + def test_lazily_resolved_names_are_still_callable(self) -> None: + from molmcp import Provider, create_plane, create_stack, discover_providers + + resolved: list[tuple[str, object]] = [ + ("create_plane", create_plane), + ("create_stack", create_stack), + ("Provider", Provider), + ("discover_providers", discover_providers), + ] + + assert [name for name, obj in resolved if not callable(obj)] == [] + + def test_unknown_attribute_raises_and_dir_lists_the_public_names(self) -> None: + module_getattr = getattr(molmcp, "__getattr__", None) + + assert callable(module_getattr), ( + "molmcp must define a PEP 562 module __getattr__ to resolve its " + "public names on first use" + ) + with pytest.raises(AttributeError): + module_getattr("no_such_name") + assert set(dir(molmcp)) >= set(molmcp.__all__) + + def test_worker_facade_body_is_only_a_lazy_reexport(self) -> None: + described = [_describe(node) for node in _module_body(WORKER_INIT)] + + assert described[:3] == ["docstring", "future-import", "__all__"], described + assert sorted(described[3:]) == ["def __dir__", "def __getattr__"], described + + def test_worker_facade_defines_no_worker_and_imports_no_sibling(self) -> None: + tree = _parse(WORKER_INIT) + + classes = [ + node.name + for node in ast.walk(tree) + if isinstance(node, ast.ClassDef) and node.name == "WorkerProvider" + ] + # ``.worker`` is banned in the module body only: __getattr__ is exactly + # where ``from .worker import WorkerProvider`` is supposed to happen. + body_imports = sorted( + str(node.module) + for node in tree.body + if isinstance(node, ast.ImportFrom) + and node.level == 1 + and node.module in {"proxy", "supervisor", "worker"} + ) + sibling_imports = sorted( + str(node.module) + for node in ast.walk(tree) + if isinstance(node, ast.ImportFrom) + and node.level == 1 + and node.module in {"proxy", "supervisor"} + ) + + assert classes == [], "WorkerProvider lives in worker.py, not the façade" + assert body_imports == [], body_imports + assert sibling_imports == [], sibling_imports + assert not _imports_fastmcp(tree), ( + "the façade must stay importable from a child process that has no " + "FastMCP loaded" + ) + + def test_importing_the_protocol_module_loads_no_fastmcp(self) -> None: + result = subprocess.run( + [sys.executable, "-c", _ISOLATION_PROBE], + capture_output=True, + text=True, + cwd=REPO_ROOT, + env={**os.environ, "PYTHONPATH": str(SRC)}, + check=False, + ) + + assert result.returncode == 0, result.stderr + printed = result.stdout.splitlines() + assert printed and printed[-1] == "[]", ( + f"importing molmcp.provider_worker.protocol loaded FastMCP: " + f"{result.stdout!r}" + ) + + def test_version_still_comes_from_the_distribution_metadata(self) -> None: + assert molmcp.__version__ == importlib.metadata.version("molcrafts-molmcp") diff --git a/tests/test_mcp_vnext.py b/tests/test_mcp_vnext.py index 5aba0f6..3ad5901 100644 --- a/tests/test_mcp_vnext.py +++ b/tests/test_mcp_vnext.py @@ -10,6 +10,8 @@ from molmcp.planes import route_task _CORE_TOOLS = { + "list_planes", + "route", "info", "packages", "outline", @@ -91,23 +93,41 @@ def _template_uri(t) -> str: _ = quote -async def test_catalog_plane_lists_and_routes(): - catalog = create_plane("catalog") - tools = await catalog.list_tools() - names = {t.name for t in tools} - assert names == {"list_planes", "route"} - - planes = await call(catalog, "list_planes") +async def test_core_lists_and_routes(server): + planes = await call(server, "list_planes") assert planes["ok"] is True + assert planes["core"] == "molcrafts" ids = {p["id"] for p in planes["planes"]} - assert "catalog" in ids and "molcrafts" in ids + assert "molcrafts" in ids + assert "catalog" not in ids + core = next(p for p in planes["planes"] if p["id"] == "molcrafts") + assert core["kind"] == "core" + assert core["disableable"] is False - routed = await call(catalog, "route", {"task": "draw dopamine in the viewer"}) + routed = await call(server, "route", {"task": "draw dopamine in the viewer"}) assert any(m["plane"] == "molvis" for m in routed["planes"]) - # Pure function path matches tool + assert routed["core"] == "molcrafts" + assert all(m["plane"] != "molcrafts" for m in routed["planes"]) assert route_task("submit a slurm job")["planes"][0]["plane"] == "molq" +def test_catalog_plane_is_gone(): + import pytest + + with pytest.raises(ValueError, match="catalog is not a plane"): + create_plane("catalog") + + +def test_route_task_does_not_emit_core(): + knowledge = route_task("how to import a symbol from the package docs") + assert knowledge["core"] == "molcrafts" + assert knowledge["planes"] == [] + drawing = route_task("draw dopamine") + assert [m["plane"] for m in drawing["planes"]] == ["molvis"] + assert drawing["namespaces"] == ["molvis"] + assert drawing["serve_commands"] == ["molmcp serve"] + + async def test_multi_provider_server_rejected(): from fastmcp import FastMCP from mcp.types import ToolAnnotations @@ -130,5 +150,5 @@ def t() -> str: import pytest - with pytest.raises(ValueError, match="multi-provider"): + with pytest.raises(ValueError, match="create_stack"): create_plane("a", providers=[P1(), P2()], discover_entry_points=False) diff --git a/tests/test_middleware/test_naming.py b/tests/test_middleware/test_naming.py index 02ac764..fdbd25e 100644 --- a/tests/test_middleware/test_naming.py +++ b/tests/test_middleware/test_naming.py @@ -67,9 +67,7 @@ def molexp_molexp_oops() -> str: assert_plane_tool_names(mcp, "molexp") -def test_builtin_planes_pass_naming(): - create_plane("catalog") - # molcrafts with empty collection +def test_core_plane_passes_naming(): from molmcp import CollectionIndex create_plane( diff --git a/tests/test_no_builtin_harness_source.py b/tests/test_no_builtin_harness_source.py new file mode 100644 index 0000000..3fbf740 --- /dev/null +++ b/tests/test_no_builtin_harness_source.py @@ -0,0 +1,84 @@ +"""No harness source is built in, and ``components/`` never hears of one. + +molmcp serves components from the harness repositories its operator named, +and from no others. The guarantee worth testing is not that some list of +literals is absent from the source — it is that an install which names +nothing gets nothing. Hence the leading assertion here: ``load_settings`` +over an empty settings tree resolves ``harness`` to the empty tuple. +``tests/test_stack.py`` (the ``harness=()`` arms) is the other half of that +criterion, recording that the empty tuple binds no store, reads no catalog +and contributes ``extras == ()``; it is not duplicated here. + +**There is deliberately no AST lint in this module.** A scan for +``HarnessSource(...)`` calls carrying string constants is defeated by +``HarnessSource(**_DEFAULT)``, by a module-level constant, and most +realistically by a module-level list of plain dicts poured through the same +``_harness_sources`` path that file data takes — which never calls +``HarnessSource(...)`` with a literal at all. A gate that cannot catch the +case it exists for is worse than none, so the behavioural assertion leads +and nothing lints behind it. A bare literal blocklist on ``"molcrafts"`` is +refused for a second reason: that string is the core plane id and appears +throughout ``server.py`` for unrelated reasons. + +The second guard is a boundary. ``components/`` is a shared stdlib leaf, +admitted only when an inner layer needs it; a harness source is a +``settings.py`` concept and nothing in ``components/`` has any reason to +know one exists. ``ComponentSpec``'s id grammar is deliberately *not* +re-asserted here — ``tests/test_components/test_models.py`` +``TestComponentSpec.test_rejects_id_mismatch`` owns "an id that is not +``f'{kind}.{name}'`` raises ``CatalogError``", and cross-source namespacing +is out of scope for the spec that added this module. + +Both guards are expected to be green on arrival: their job is to fail +*later*, if someone builds an official coordinate in or teaches +``components/`` about settings. This lives in a module of its own rather +than inside ``tests/test_settings.py`` because it reads other modules' +source as data, which is not ``settings.py`` behaviour; +``tests/test_no_env_switches.py`` is the repo's existing pattern for a +repo-wide structural assertion housed this way. +""" + +from __future__ import annotations + +import ast +from pathlib import Path + +import pytest + +from molmcp import settings as st + +SRC = Path(__file__).resolve().parents[1] / "src" / "molmcp" + +#: The components layer, which must not learn that a harness source exists. +_COMPONENT_MODULES = ( + SRC / "components" / "models.py", + SRC / "components" / "catalog.py", + SRC / "components" / "locator.py", +) + +#: Naming either of these in ``components/`` means the boundary moved. +_HARNESS_NAMES = ("HarnessSource", "harness_source") + + +def test_an_empty_settings_tree_names_no_harness_source(home: Path, tmp_path: Path): + """No file, no source: the empty tuple is the un-harnessed install.""" + assert st.load_settings(tmp_path / "repo").harness == () + + +@pytest.mark.parametrize("name", _HARNESS_NAMES) +@pytest.mark.parametrize("path", _COMPONENT_MODULES, ids=lambda p: p.name) +def test_the_components_layer_never_names_a_harness_source(path: Path, name: str): + assert name not in path.read_text(encoding="utf-8"), ( + f"{path.relative_to(SRC)} names {name}. A harness source is a settings " + f"concept; components/ is a shared leaf that must not depend on it. " + f"Cross-source namespacing belongs to the resolution layer, keyed by a " + f"(source_name, component_id) pair, and never enters ComponentSpec.id." + ) + + +@pytest.mark.parametrize("path", _COMPONENT_MODULES, ids=lambda p: p.name) +def test_a_guarded_module_is_still_a_live_module_under_src(path: Path): + """A renamed or deleted file would make the text guard pass vacuously.""" + assert path in set(SRC.rglob("*.py")) + + ast.parse(path.read_text(encoding="utf-8")) diff --git a/tests/test_no_env_switches.py b/tests/test_no_env_switches.py index c5a24fd..1c5ea5e 100644 --- a/tests/test_no_env_switches.py +++ b/tests/test_no_env_switches.py @@ -16,6 +16,7 @@ from pathlib import Path import pytest +from _ast_checks import reads_environment SRC = Path(__file__).resolve().parents[1] / "src" / "molmcp" @@ -31,22 +32,11 @@ } -def _reads_environment(tree: ast.AST) -> bool: - for node in ast.walk(tree): - if isinstance(node, ast.Attribute) and node.attr in {"environ", "getenv"}: - value = node.value - if isinstance(value, ast.Name) and value.id == "os": - return True - if isinstance(node, ast.Name) and node.id == "getenv": - return True - return False - - @pytest.mark.parametrize("path", sorted(SRC.rglob("*.py")), ids=lambda p: p.name) def test_no_module_reads_the_environment_for_configuration(path: Path): tree = ast.parse(path.read_text(encoding="utf-8")) - if not _reads_environment(tree): + if not reads_environment(tree): return assert path.relative_to(SRC).as_posix() in _ALLOWED, ( @@ -62,7 +52,7 @@ def test_the_allowlist_does_not_rot(): name for name in _ALLOWED if not (SRC / name).is_file() - or not _reads_environment(ast.parse((SRC / name).read_text())) + or not reads_environment(ast.parse((SRC / name).read_text())) ] assert stale == [] diff --git a/tests/test_planes.py b/tests/test_planes.py new file mode 100644 index 0000000..d546dbf --- /dev/null +++ b/tests/test_planes.py @@ -0,0 +1,314 @@ +"""The plane catalog: membership from the group, product copy from here. + +``list_plane_infos`` / ``known_plane_ids`` answer *which MCP servers this +install can offer*. That question has exactly one authority — the +``molmcp.providers`` entry-point group, read through +``discover_providers``. A name that only a table inside ``planes.py`` knows +about is a second authority: it makes the catalog advertise a plane that +``molmcp serve`` cannot start. + +Two jobs stay in this module and are pinned here as well, because they are +what makes deleting the membership table safe rather than lossy: + +* the **copy table** — product ``purpose`` / ``when_to_connect`` sentences, + keyed by name but never a source of membership; +* ``tools_hint`` — read off the discovered instance's own ``tool_specs()``, + duck-typed exactly like ``probe`` is, so no parallel tool list exists. + +Discovery is faked with ``monkeypatch``: no entry point is added to +``pyproject.toml`` for a fixture, and nothing here imports +``molmcp.providers.base`` — a plain object with a ``tool_specs`` method is +the whole contract the catalog may rely on. +""" + +from __future__ import annotations + +import ast +from collections.abc import Iterator +from dataclasses import dataclass +from pathlib import Path + +import pytest + +from molmcp import planes + +#: Today's product sentences, keyed by name. Holding a key here is *not* +#: membership: only a name the group reported may be looked up in it. +_COPY: dict[str, tuple[str, str]] = { + "molvis": ( + "Live molvis viewer: persistent Python namespace + browser canvas.", + "User wants to draw, load, select, or interact with a molecule in 3D.", + ), + "molq": ( + "molq job lifecycle: list/get/logs destinations; opt-in submit/cancel.", + "User wants cluster jobs, queue status, or submission.", + ), + "molexp": ( + "molexp workspace navigation, idempotent scaffold, and adoption of a " + "legacy data directory (not a run driver).", + "User works with experiment workspaces, projects, FAIR layout, or has " + "a folder of results to lift into one.", + ), +} + +#: What a discovered name the copy table never heard of has to read like. +_GENERIC_PURPOSE = "Provider plane 'demo' (entry point molmcp.providers)." +_GENERIC_WHEN = "When work needs the 'demo' product surface." + + +@dataclass(frozen=True, slots=True) +class _Spec: + """The single field ``tools_hint`` reads off a published tool spec.""" + + name: str + + +class _Plane: + """A discovered plane object that publishes its own tool specs.""" + + def __init__(self, name: str, *tools: str, available: bool = True) -> None: + self.name = name + self._tools = tools + self._available = available + + def probe(self) -> bool: + return self._available + + def register(self, mcp: object) -> None: + raise AssertionError(f"listing planes must not register {self.name!r}") + + def tool_specs(self) -> Iterator[_Spec]: + return iter([_Spec(name=tool) for tool in self._tools]) + + +class _BarePlane: + """The Protocol minimum: a name and ``register``, no ``tool_specs``.""" + + def __init__(self, name: str, *, available: bool = True) -> None: + self.name = name + self._available = available + + def probe(self) -> bool: + return self._available + + def register(self, mcp: object) -> None: + raise AssertionError(f"listing planes must not register {self.name!r}") + + +def _discover( + monkeypatch: pytest.MonkeyPatch, + *members: _Plane | _BarePlane, +) -> list[bool]: + """Make the group report *members*; record every ``only_available`` asked. + + The fake filters on ``probe()`` itself, the way the real + ``discover_providers`` does, so an unavailable member is what the + catalog never sees rather than something the catalog has to skip. + """ + asked: list[bool] = [] + + def discover_providers( + *, + failures: list[dict[str, str]] | None = None, + only_available: bool = False, + ) -> list[_Plane | _BarePlane]: + asked.append(only_available) + return [m for m in members if not only_available or m.probe()] + + monkeypatch.setattr(planes, "discover_providers", discover_providers) + return asked + + +def _copy_cases() -> list[tuple[str, str, str]]: + """One ``(name, purpose, when)`` case per row of the copy table.""" + return [(name, purpose, when) for name, (purpose, when) in _COPY.items()] + + +def _ids(infos: list[planes.PlaneInfo]) -> list[str]: + return [info.id for info in infos] + + +def _one(infos: list[planes.PlaneInfo], plane_id: str) -> planes.PlaneInfo: + """The single listed plane called *plane_id* — listed exactly once.""" + matches = [info for info in infos if info.id == plane_id] + assert len(matches) == 1, f"{plane_id!r} listed {len(matches)} times" + return matches[0] + + +class TestListPlaneInfos: + """One row per discovered plane, plus the always-on core.""" + + @pytest.mark.parametrize(("name", "purpose", "when"), _copy_cases()) + def test_an_official_name_gets_the_copy_table_sentences( + self, monkeypatch, name: str, purpose: str, when: str + ): + """A discovered official plane still reads as the product, not a stub. + + These three sentences have no other home: dropping them would leave + molvis / molq / molexp describing themselves as "the 'molvis' product + surface", which is what the generic fallback is *for*. + """ + _discover(monkeypatch, _Plane(name, "open")) + + info = _one(planes.list_plane_infos(), name) + + assert info.purpose == purpose + assert info.when_to_connect == when + + def test_an_official_name_hints_the_tools_its_instance_publishes(self, monkeypatch): + """``tools_hint`` is this instance's ``tool_specs()``, not a copy of it.""" + _discover(monkeypatch, _Plane("molvis", "open")) + + info = _one(planes.list_plane_infos(), "molvis") + + assert info.tools_hint == ("open",) + + def test_an_unknown_member_gets_the_generic_sentences(self, monkeypatch): + """A name outside the copy table is a member, described generically.""" + _discover(monkeypatch, _Plane("demo", "peek")) + + info = _one(planes.list_plane_infos(), "demo") + + assert info.purpose == _GENERIC_PURPOSE + assert info.when_to_connect == _GENERIC_WHEN + + def test_an_unknown_member_hints_the_tools_its_instance_publishes( + self, monkeypatch + ): + """No copy-table row, yet the tools are known — they come off the object.""" + _discover(monkeypatch, _Plane("demo", "peek")) + + info = _one(planes.list_plane_infos(), "demo") + + assert info.tools_hint == ("peek",) + + def test_a_member_without_tool_specs_hints_no_tools(self, monkeypatch): + """``tool_specs`` is optional, like ``probe``: absent means no hint.""" + bare = _BarePlane("molq") + assert not hasattr(bare, "tool_specs") + _discover(monkeypatch, bare) + + info = _one(planes.list_plane_infos(), "molq") + + assert info.tools_hint == () + + def test_the_copy_table_still_answers_a_member_without_tool_specs( + self, monkeypatch + ): + """Copy is keyed by name; it does not depend on publishing tools.""" + _discover(monkeypatch, _BarePlane("molq")) + + info = _one(planes.list_plane_infos(), "molq") + + assert (info.purpose, info.when_to_connect) == _COPY["molq"] + + @pytest.mark.parametrize("include_unavailable", [False, True]) + def test_an_empty_group_leaves_only_the_core( + self, monkeypatch, include_unavailable: bool + ): + """Nothing installed lists nothing, though the copy table is full.""" + _discover(monkeypatch) + + infos = planes.list_plane_infos( + include_unavailable_providers=include_unavailable + ) + + assert _ids(infos) == [planes.CORE_PLANE_ID] + + def test_unavailable_members_are_listed_when_diagnostics_ask(self, monkeypatch): + """A discovered plane whose science package is missing is still real.""" + _discover(monkeypatch, _Plane("demo", "peek", available=False)) + + infos = planes.list_plane_infos(include_unavailable_providers=True) + + assert _ids(infos) == [planes.CORE_PLANE_ID, "demo"] + + def test_an_official_name_that_was_not_discovered_is_not_invented( + self, monkeypatch + ): + """Diagnostics widen ``probe()``, never the membership question.""" + _discover(monkeypatch, _Plane("demo", "peek", available=False)) + + listed = _ids(planes.list_plane_infos(include_unavailable_providers=True)) + + for name in _COPY: + assert name not in listed + + def test_diagnostics_ask_discovery_for_the_unavailable_members(self, monkeypatch): + """The wider list comes from ``only_available=False``, not from a table.""" + asked = _discover(monkeypatch, _Plane("demo", "peek", available=False)) + + planes.list_plane_infos(include_unavailable_providers=True) + + assert False in asked + + def test_the_module_has_no_provider_meta_membership_table(self): + """The three-jobs table is gone: membership, copy, and tools split up.""" + assert not hasattr(planes, "_PROVIDER_META") + + def test_the_module_never_imports_the_provider_base_module(self): + """``tools_hint`` is duck-typed; layer 2 does not depend on the SDK base.""" + source = Path(planes.__file__).read_text(encoding="utf-8") + assert "providers.base" not in source + imported: list[str] = [] + for node in ast.walk(ast.parse(source)): + if isinstance(node, ast.ImportFrom): + imported.append(node.module or "") + elif isinstance(node, ast.Import): + imported += [alias.name for alias in node.names] + assert [name for name in imported if "providers" in name] == [] + + +class TestKnownPlaneIds: + """What ``molmcp serve `` may accept — the same one authority.""" + + def test_available_ids_are_the_core_plus_what_discovery_reported(self, monkeypatch): + _discover(monkeypatch, _Plane("demo", "peek")) + + assert planes.known_plane_ids(only_available=True) == frozenset( + {planes.CORE_PLANE_ID, "demo"} + ) + + def test_unavailable_ids_come_from_discovery_not_from_the_copy_table( + self, monkeypatch + ): + """Explicit serve is allowed to fail loudly — but only for real planes.""" + _discover(monkeypatch, _Plane("demo", "peek", available=False)) + + assert planes.known_plane_ids() == frozenset({planes.CORE_PLANE_ID, "demo"}) + + @pytest.mark.parametrize("only_available", [False, True]) + def test_an_empty_group_knows_only_the_core( + self, monkeypatch, only_available: bool + ): + _discover(monkeypatch) + + assert planes.known_plane_ids(only_available=only_available) == frozenset( + {planes.CORE_PLANE_ID} + ) + + @pytest.mark.parametrize("only_available", [False, True]) + def test_the_only_available_flag_reaches_discovery_unchanged( + self, monkeypatch, only_available: bool + ): + asked = _discover(monkeypatch, _Plane("demo", "peek")) + + planes.known_plane_ids(only_available=only_available) + + assert asked == [only_available] + + +class TestRouteTask: + """Keyword routing is a core table, not a membership list.""" + + def test_drawing_still_routes_to_molvis(self): + answer = planes.route_task("draw dopamine") + + assert [match["plane"] for match in answer["planes"]] == ["molvis"] + + def test_an_unknown_member_is_listed_but_never_keyword_routed(self, monkeypatch): + """Joining the group publishes a plane; it does not claim vocabulary.""" + _discover(monkeypatch, _Plane("demo", "peek")) + + assert "demo" in _ids(planes.list_plane_infos()) + assert planes.route_task("demo peek please")["planes"] == [] diff --git a/tests/test_provider/test_provider.py b/tests/test_provider/test_provider.py index 7ddce9c..82945b2 100644 --- a/tests/test_provider/test_provider.py +++ b/tests/test_provider/test_provider.py @@ -13,6 +13,7 @@ ) from molmcp import provider as provider_module from molmcp.middleware import MissingAnnotationsError +from molmcp.provider_sdk import ProviderBase def _server(*, provider, **kwargs): @@ -105,6 +106,44 @@ def load(): ] +def test_discover_providers_accepts_sdk_provider_base(monkeypatch): + """A public-SDK plane is loaded; the entry point still owns the name.""" + + class SdkPlane(ProviderBase): + name = "sdkplane" + + class Matching: + name = "sdkplane" + + @staticmethod + def load(): + return SdkPlane + + class Mismatched: + name = "declared" + + @staticmethod + def load(): + return SdkPlane + + monkeypatch.setattr( + provider_module.importlib.metadata, + "entry_points", + lambda **kwargs: [Matching(), Mismatched()], + ) + failures: list[dict[str, str]] = [] + found = discover_providers(failures=failures) + assert [provider.name for provider in found] == ["sdkplane"] + assert type(found[0]) is SdkPlane + assert failures == [ + { + "entry_point": "declared", + "phase": "authority", + "error_type": "NamespaceMismatch", + } + ] + + def test_only_available_silently_omits_failed_probe(monkeypatch): """Runtime catalog omit — not a pytest.skip.""" diff --git a/tests/test_provider_sdk.py b/tests/test_provider_sdk.py new file mode 100644 index 0000000..ffb8e39 --- /dev/null +++ b/tests/test_provider_sdk.py @@ -0,0 +1,315 @@ +"""Public Provider SDK — the surface a plane author imports.""" + +from __future__ import annotations + +import asyncio +import dataclasses +import importlib +import importlib.util +import sys +from importlib.machinery import ModuleSpec + +import pytest +from fastmcp import FastMCP + +import molmcp.provider_sdk as sdk +from molmcp.provider import Provider as protocol +from molmcp.provider_sdk import ( + APPEND_WRITE, + IDEMPOTENT_WRITE, + LOCAL_MUTATION, + MUTATION, + READ_ONLY, + READ_REMOTE, + Provider, + ProviderBase, + ToolSpec, + tool, +) +from molmcp.providers import annotations as legacy_annotations +from molmcp.providers.base import ProviderBase as LegacyProviderBase +from molmcp.providers.base import ToolSpec as LegacyToolSpec +from molmcp.providers.base import tool as legacy_tool + +_SDK_EXPORTS = [ + "APPEND_WRITE", + "IDEMPOTENT_WRITE", + "LOCAL_MUTATION", + "MUTATION", + "Provider", + "ProviderBase", + "READ_ONLY", + "READ_REMOTE", + "ToolSpec", + "tool", +] + + +class TestToolSpec: + def test_is_frozen(self): + spec = ToolSpec(name="peek", annotations=READ_ONLY, attribute="peek") + with pytest.raises(dataclasses.FrozenInstanceError): + spec.name = "open" # type: ignore[misc] + + def test_uses_slots(self): + spec = ToolSpec(name="peek", annotations=READ_ONLY, attribute="peek") + assert not hasattr(spec, "__dict__") + assert hasattr(ToolSpec, "__slots__") + + +class TestToolDecorator: + def test_default_wire_name_is_the_method_name(self): + class Demo(ProviderBase): + name = "demo" + + @tool(READ_ONLY) + def peek(self) -> dict[str, bool]: + """Look.""" + return {"ok": True} + + specs = list(Demo().tool_specs()) + assert [spec.name for spec in specs] == ["peek"] + assert specs[0].attribute == "peek" + + def test_explicit_wire_name_is_bare(self): + class Demo(ProviderBase): + name = "demo" + + @tool(READ_ONLY, name="open") + def open_thing(self) -> dict[str, bool]: + """Open.""" + return {"ok": True} + + specs = list(Demo().tool_specs()) + assert [spec.name for spec in specs] == ["open"] + assert specs[0].attribute == "open_thing" + + +class TestProviderBase: + def test_tool_specs_are_base_first_in_declaration_order(self): + class Base(ProviderBase): + name = "demo" + + @tool(READ_ONLY) + def alpha(self) -> dict[str, str]: + """A.""" + return {"id": "alpha"} + + @tool(READ_ONLY) + def beta(self) -> dict[str, str]: + """B.""" + return {"id": "beta"} + + class Child(Base): + @tool(READ_ONLY) + def gamma(self) -> dict[str, str]: + """C.""" + return {"id": "gamma"} + + specs = list(Child().tool_specs()) + assert [spec.attribute for spec in specs] == ["alpha", "beta", "gamma"] + assert [spec.name for spec in specs] == ["alpha", "beta", "gamma"] + + def test_overriding_an_attribute_replaces_the_spec(self): + class Base(ProviderBase): + name = "demo" + + @tool(READ_ONLY) + def peek(self) -> dict[str, str]: + """Original.""" + return {"id": "base"} + + class Child(Base): + @tool(MUTATION) + def peek(self) -> dict[str, str]: + """Replaced.""" + return {"id": "child"} + + specs = list(Child().tool_specs()) + assert len(specs) == 1 + assert specs[0].attribute == "peek" + assert specs[0].annotations is MUTATION + + def test_register_attaches_declared_tools(self): + class Demo(ProviderBase): + name = "demo" + + @tool(READ_ONLY) + def ping(self) -> dict[str, bool]: + """Ping the plane.""" + return {"ok": True} + + @tool(READ_ONLY, name="open") + def open_thing(self) -> dict[str, bool]: + """Open.""" + return {"ok": True} + + mcp = FastMCP("demo") + Demo().register(mcp) + names = {item.name for item in asyncio.run(mcp.list_tools())} + assert names == {"ping", "open"} + + def test_duplicate_wire_names_are_rejected(self): + class Clashing(ProviderBase): + name = "clash" + + @tool(READ_ONLY, name="thing") + def first(self) -> dict[str, int]: + """One.""" + return {"n": 1} + + @tool(READ_ONLY, name="thing") + def second(self) -> dict[str, int]: + """Two.""" + return {"n": 2} + + with pytest.raises(ValueError) as excinfo: + Clashing().register(FastMCP("clash")) + + message = str(excinfo.value) + assert "thing" in message + assert "first" in message + assert "second" in message + + def test_probe_is_true_without_upstream(self): + class Demo(ProviderBase): + name = "demo" + + assert Demo().probe() is True + + def test_probe_is_false_when_upstream_is_missing(self): + class Absent(ProviderBase): + name = "absent" + upstream = "molcrafts-nope" + import_name = "molmcp_sdk_nope_xyz" + + assert Absent().probe() is False + + def test_probe_is_true_when_upstream_is_installed(self): + class Present(ProviderBase): + name = "present" + upstream = "pytest" + import_name = "pytest" + + assert Present().probe() is True + + def test_missing_upstream_names_the_install_command(self): + class Absent(ProviderBase): + name = "absent" + upstream = "molcrafts-nope" + import_name = "molmcp_sdk_nope_xyz" + + with pytest.raises(RuntimeError) as excinfo: + Absent().require_upstream() + + message = str(excinfo.value) + assert "molcrafts-nope" in message + assert "pip install molcrafts-nope" in message + + +class TestAnnotationVocabulary: + @pytest.mark.parametrize( + ("constant", "read_only", "destructive", "idempotent", "open_world"), + [ + (READ_ONLY, True, False, True, False), + (READ_REMOTE, True, False, False, True), + (MUTATION, False, True, False, True), + (LOCAL_MUTATION, False, True, True, False), + (APPEND_WRITE, False, False, False, False), + (IDEMPOTENT_WRITE, False, False, True, False), + ], + ids=[ + "READ_ONLY", + "READ_REMOTE", + "MUTATION", + "LOCAL_MUTATION", + "APPEND_WRITE", + "IDEMPOTENT_WRITE", + ], + ) + def test_constant_states_all_four_hints( + self, + constant, + read_only: bool, + destructive: bool, + idempotent: bool, + open_world: bool, + ): + assert constant.read_only_hint is read_only + assert constant.destructive_hint is destructive + assert constant.idempotent_hint is idempotent + assert constant.open_world_hint is open_world + + +class TestLegacyProviderImports: + def test_tool_spec_is_the_same_object(self): + assert ToolSpec is LegacyToolSpec + + def test_provider_base_is_the_same_object(self): + assert ProviderBase is LegacyProviderBase + + def test_tool_is_the_same_object(self): + assert tool is legacy_tool + + @pytest.mark.parametrize( + "name", + [ + "APPEND_WRITE", + "IDEMPOTENT_WRITE", + "LOCAL_MUTATION", + "MUTATION", + "READ_ONLY", + "READ_REMOTE", + ], + ) + def test_annotation_constant_is_the_same_object(self, name: str): + assert getattr(sdk, name) is getattr(legacy_annotations, name) + + +class TestSdkExports: + def test_all_is_exactly_the_sorted_public_names(self): + assert _SDK_EXPORTS == sorted(_SDK_EXPORTS) + assert sdk.__all__ == _SDK_EXPORTS + + def test_provider_is_the_runtime_protocol(self): + assert Provider is protocol + + +class TestImportSafety: + def test_importing_the_sdk_does_not_load_science_packages(self): + science = ("molvis", "molq", "molexp") + held_science = {name: sys.modules.pop(name, None) for name in science} + held_sdk = sys.modules.pop("molmcp.provider_sdk", None) + try: + importlib.import_module("molmcp.provider_sdk") + for name in science: + assert name not in sys.modules + finally: + if held_sdk is not None: + sys.modules["molmcp.provider_sdk"] = held_sdk + for name, previous in held_science.items(): + if previous is not None: + sys.modules[name] = previous + else: + sys.modules.pop(name, None) + + def test_probe_uses_find_spec_and_does_not_import_the_module( + self, monkeypatch: pytest.MonkeyPatch + ): + seen: list[str] = [] + + def fake_find_spec(name: str, _package: str | None = None) -> ModuleSpec | None: + seen.append(name) + return None + + monkeypatch.setattr(importlib.util, "find_spec", fake_find_spec) + + class Demo(ProviderBase): + name = "demo" + upstream = "molcrafts-nope" + import_name = "molmcp_sdk_probe_absent" + + sys.modules.pop("molmcp_sdk_probe_absent", None) + assert Demo().probe() is False + assert seen == ["molmcp_sdk_probe_absent"] + assert "molmcp_sdk_probe_absent" not in sys.modules diff --git a/tests/test_provider_worker/__init__.py b/tests/test_provider_worker/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/tests/test_provider_worker/fixtures/echo.py b/tests/test_provider_worker/fixtures/echo.py new file mode 100644 index 0000000..debebbb --- /dev/null +++ b/tests/test_provider_worker/fixtures/echo.py @@ -0,0 +1,16 @@ +"""Minimal in-process provider used to drive the worker subprocess.""" + +from __future__ import annotations + +from molmcp.provider_sdk import READ_ONLY, ProviderBase, tool + + +class EchoProvider(ProviderBase): + """Echo plane — one read-only tool.""" + + name = "echo" + + @tool(READ_ONLY) + def echo(self, text: str) -> dict[str, str]: + """Echo text back.""" + return {"text": text} diff --git a/tests/test_provider_worker/test_child.py b/tests/test_provider_worker/test_child.py new file mode 100644 index 0000000..bc0ee9e --- /dev/null +++ b/tests/test_provider_worker/test_child.py @@ -0,0 +1,336 @@ +"""child.py — the path-launched worker script, driven as a real subprocess. + +Every live test here spawns the script a Supervisor spawns +(``sys.executable -P child.py --entrypoint echo:EchoProvider --path +``) and speaks duplex v1 over its stdio. The child is a *script*: +the package has no ``__main__.py`` and the launch vector carries no ``-m``. + +The static tests guard what the child must never grow: a fastmcp import, an +``instance.register(mcp)`` call, a faked ``molmcp`` module, a +``spec_from_file_location`` loader, environment-driven configuration, or a +hand-written JSON Schema. ``hello`` carries signature *facts*; FastMCP +produces the schema in the parent, from the rebuilt callable. +""" + +from __future__ import annotations + +import ast +import contextlib +import os +import subprocess +import sys +import threading +from collections.abc import Iterator +from pathlib import Path + +from _ast_checks import reads_environment + +from molmcp.provider_worker.protocol import decode, encode_invoke, encode_shutdown + +_REPO = Path(__file__).resolve().parents[2] +_SRC = _REPO / "src" +_CHILD = _SRC / "molmcp" / "provider_worker" / "child.py" +_MAIN = _SRC / "molmcp" / "provider_worker" / "__main__.py" +_FIXTURES = Path(__file__).parent / "fixtures" + +#: The one launch vector: a filesystem path, never ``python -m``. ``-P`` keeps +#: the script's own directory out of ``sys.path``, so ``--path`` is the only +#: root the fixture can be imported from. +_ARGV = [ + sys.executable, + "-P", + str(_CHILD), + "--entrypoint", + "echo:EchoProvider", + "--path", + str(_FIXTURES), +] + +#: Seconds a single read or exit may take before the test fails instead of +#: wedging the suite behind a hung child. +_TIMEOUT = 15.0 + +#: Literals that would mean the child hand-rolled a JSON Schema. +_SCHEMA_LITERALS = ("properties", "inputSchema", "additionalProperties", "$schema") + + +def _environment() -> dict[str, str]: + """The parent environment plus ``src`` on ``PYTHONPATH``. + + The child imports ``molmcp`` for real; nothing here configures it. + """ + return {**os.environ, "PYTHONPATH": str(_SRC)} + + +@contextlib.contextmanager +def _child_process() -> Iterator[subprocess.Popen[str]]: + """Spawn the real child script, reaping it however the test leaves it.""" + process = subprocess.Popen( + list(_ARGV), + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + text=True, + bufsize=1, + env=_environment(), + ) + try: + yield process + finally: + _reap(process) + + +def _reap(process: subprocess.Popen[str]) -> None: + """Terminate the child and close its pipes, whatever state it is in.""" + if process.poll() is None: + process.terminate() + try: + process.wait(timeout=_TIMEOUT) + except subprocess.TimeoutExpired: + process.kill() + process.wait(timeout=_TIMEOUT) + for stream in (process.stdin, process.stdout): + if stream is not None: + with contextlib.suppress(OSError, ValueError): + stream.close() + + +def _readline(process: subprocess.Popen[str]) -> str: + """One NDJSON line from the child, failing the test if it never comes.""" + stdout = process.stdout + if stdout is None: + raise AssertionError("child was spawned without a stdout pipe") + lines: list[str] = [] + + def read() -> None: + lines.append(stdout.readline()) + + reader = threading.Thread(target=read, daemon=True) + reader.start() + reader.join(_TIMEOUT) + if reader.is_alive(): + raise AssertionError(f"child wrote no line within {_TIMEOUT}s") + if not lines[0]: + raise AssertionError("child closed stdout instead of answering") + return lines[0] + + +def _send(process: subprocess.Popen[str], line: str) -> None: + """Write one NDJSON line to the child and flush it.""" + stdin = process.stdin + if stdin is None: + raise AssertionError("child was spawned without a stdin pipe") + stdin.write(line) + stdin.flush() + + +def _source() -> str: + """The child script as text; it must exist to be launchable by path.""" + if not _CHILD.is_file(): + raise AssertionError(f"{_CHILD} does not exist") + return _CHILD.read_text(encoding="utf-8") + + +def _imported_modules(tree: ast.Module) -> set[str]: + """Every dotted module name the child imports, in either form.""" + names: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + names.update(alias.name for alias in node.names) + elif isinstance(node, ast.ImportFrom) and node.module is not None: + names.add(node.module) + return names + + +def _called_names(tree: ast.Module) -> set[str]: + """Every name called in the child, bare or as an attribute.""" + names: set[str] = set() + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + if isinstance(node.func, ast.Name): + names.add(node.func.id) + elif isinstance(node.func, ast.Attribute): + names.add(node.func.attr) + return names + + +def _string_constants(tree: ast.Module) -> set[str]: + """Every string literal in the child.""" + return { + node.value + for node in ast.walk(tree) + if isinstance(node, ast.Constant) and isinstance(node.value, str) + } + + +class TestChild: + """The worker child script, spoken to over duplex v1.""" + + # -- hello --------------------------------------------------------- + + def test_hello_is_the_first_line_and_names_the_plane(self): + """The child announces itself before anything is asked of it.""" + with _child_process() as process: + hello = decode(_readline(process)) + + assert hello["type"] == "hello" + assert hello["protocol"] == 1 + assert hello["name"] == "echo" + + def test_hello_declares_exactly_the_bare_echo_tool(self): + """One tool, wire-named ``echo`` — never a namespaced ``echo_echo``.""" + with _child_process() as process: + hello = decode(_readline(process)) + + assert {spec["name"] for spec in hello["tools"]} == {"echo"} + + def test_hello_carries_the_method_docstring(self): + """``doc`` is the method's docstring; the parent uses it as-is.""" + with _child_process() as process: + hello = decode(_readline(process)) + + (spec,) = hello["tools"] + assert spec["doc"] == "Echo text back." + + def test_hello_carries_the_four_annotation_hints(self): + """READ_ONLY reaches the parent as four JSON booleans, not an object.""" + with _child_process() as process: + hello = decode(_readline(process)) + + (spec,) = hello["tools"] + assert spec["annotations"] == { + "read_only_hint": True, + "destructive_hint": False, + "idempotent_hint": True, + "open_world_hint": False, + } + + def test_hello_parameters_are_signature_facts_without_self(self): + """The signature is read off the *bound* method, so ``self`` is gone.""" + with _child_process() as process: + hello = decode(_readline(process)) + + (spec,) = hello["tools"] + parameters = spec["parameters"] + assert isinstance(parameters, list) + names = [fact["name"] for fact in parameters] + assert "self" not in names + assert names == ["text"] + (text,) = parameters + assert text["kind"] == "POSITIONAL_OR_KEYWORD" + assert text["annotation"] == "str" + + # -- invoke / shutdown --------------------------------------------- + + def test_invoke_echoes_the_argument_back(self): + """An ``invoke`` frame dispatches to the method and answers ``result``.""" + with _child_process() as process: + decode(_readline(process)) + _send( + process, + encode_invoke(call_id="1", name="echo", args={"text": "ping"}), + ) + frame = decode(_readline(process)) + + assert frame["type"] == "result" + assert frame["id"] == "1" + assert frame["value"] == {"text": "ping"} + + def test_shutdown_exits_the_child_with_zero(self): + """A v1 ``shutdown`` frame ends the loop cleanly, without a signal.""" + with _child_process() as process: + decode(_readline(process)) + _send(process, encode_shutdown()) + try: + returncode = process.wait(timeout=_TIMEOUT) + except subprocess.TimeoutExpired as exc: + raise AssertionError("child ignored the v1 shutdown frame") from exc + + assert returncode == 0 + + # -- edge ---------------------------------------------------------- + + def test_unknown_tool_answers_with_an_error_frame(self): + """A name the plane does not offer is an ``error``, not a crash.""" + with _child_process() as process: + decode(_readline(process)) + _send(process, encode_invoke(call_id="7", name="nope", args={})) + frame = decode(_readline(process)) + + assert frame["type"] == "error" + assert frame["id"] == "7" + assert isinstance(frame["error"], str) + assert frame["error"] != "" + + def test_child_survives_an_error_and_serves_the_next_invoke(self): + """One bad call must not cost the supervisor its worker.""" + with _child_process() as process: + decode(_readline(process)) + _send(process, encode_invoke(call_id="7", name="nope", args={})) + assert decode(_readline(process))["type"] == "error" + _send( + process, + encode_invoke(call_id="8", name="echo", args={"text": "again"}), + ) + frame = decode(_readline(process)) + + assert frame["type"] == "result" + assert frame["id"] == "8" + assert frame["value"] == {"text": "again"} + + # -- isolation, read off the source -------------------------------- + + def test_source_never_imports_fastmcp(self): + """The whole point of the subprocess: no server library inside it.""" + source = _source() + assert "import fastmcp" not in source + assert "from fastmcp" not in source + imported = _imported_modules(ast.parse(source)) + offenders = { + name + for name in imported + if name == "fastmcp" or name.startswith("fastmcp.") + } + assert offenders == set() + + def test_source_never_calls_register(self): + """``register(mcp)`` belongs to the parent; the child only reports.""" + source = _source() + assert "register(" not in source + assert "register" not in _called_names(ast.parse(source)) + + def test_source_never_fakes_a_module(self): + """The child imports the real molmcp — no stub, no ad-hoc loader.""" + source = _source() + assert "types.ModuleType(" not in source + assert "spec_from_file_location" not in source + called = _called_names(ast.parse(source)) + assert "ModuleType" not in called + assert "spec_from_file_location" not in called + + def test_source_never_reads_the_environment(self): + """Configuration arrives on argv; the environment is only inherited.""" + source = _source() + assert "os.environ" not in source + assert "os.getenv" not in source + assert not reads_environment(ast.parse(source)) + + def test_source_never_builds_a_json_schema(self): + """Only signature facts travel; FastMCP owns the schema, in the parent.""" + source = _source() + constants = _string_constants(ast.parse(source)) + for literal in _SCHEMA_LITERALS: + assert literal not in source + assert literal not in constants + + # -- script, not module -------------------------------------------- + + def test_the_package_has_no_main_module(self): + """``python -m molmcp.provider_worker`` must stay impossible.""" + assert not _MAIN.exists() + + def test_the_launch_vector_never_uses_dash_m(self): + """The child is launched by path, with ``-P`` guarding sys.path.""" + assert "-m" not in _ARGV + assert _ARGV[1] == "-P" + assert _ARGV[2] == str(_CHILD) diff --git a/tests/test_provider_worker/test_protocol.py b/tests/test_provider_worker/test_protocol.py new file mode 100644 index 0000000..99f8571 --- /dev/null +++ b/tests/test_provider_worker/test_protocol.py @@ -0,0 +1,278 @@ +"""Duplex v1 NDJSON codec and signature facts for the provider worker.""" + +from __future__ import annotations + +import inspect + +import pytest + +from molmcp.provider_worker.protocol import ( + ANNOTATION_KEYS, + MESSAGE_TYPES, + PROTOCOL_VERSION, + ProtocolError, + decode, + encode_error, + encode_hello, + encode_invoke, + encode_result, + encode_shutdown, + rebuild_signature, + signature_facts, +) + +# One hello catalog entry, shaped exactly as child.py sends it: signature +# facts, never a JSON Schema object. +ECHO_TOOL: dict[str, object] = { + "name": "echo", + "attribute": "echo", + "doc": "Echo text back.", + "annotations": { + "read_only_hint": True, + "destructive_hint": False, + "idempotent_hint": True, + "open_world_hint": False, + }, + "parameters": [ + { + "name": "text", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "str", + "has_default": False, + } + ], +} + +# Hard-coded golden: a v0 hello frame the codec must refuse outright. +PROTOCOL_ZERO_HELLO = '{"type": "hello", "protocol": 0, "name": "echo", "tools": []}' + +FACT_KEYS = frozenset({"name", "kind", "annotation", "has_default", "default"}) +JSON_SCHEMA_KEYS = frozenset({"properties", "type", "required"}) + + +def sample(text: str, count: int = 3, *, flag: bool = False) -> dict: + """Frozen shape used to pin signature facts and their inverse.""" + return {"text": text, "count": count, "flag": flag} + + +def _sample_facts() -> list[dict[str, object]]: + return signature_facts(inspect.signature(sample)) + + +class TestProtocol: + """Unit tests for ``molmcp.provider_worker.protocol``.""" + + # --- Basics: frozen constants ------------------------------------- + + def test_protocol_version_is_one(self) -> None: + assert PROTOCOL_VERSION == 1 + + def test_message_types_are_the_five_duplex_v1_frames(self) -> None: + assert MESSAGE_TYPES == frozenset( + {"hello", "invoke", "result", "error", "shutdown"} + ) + + def test_annotation_keys_are_the_four_tool_hints(self) -> None: + assert ANNOTATION_KEYS == ( + "read_only_hint", + "destructive_hint", + "idempotent_hint", + "open_world_hint", + ) + + def test_protocol_error_is_a_runtime_error(self) -> None: + assert issubclass(ProtocolError, RuntimeError) + + # --- Basics: round trips, one frame per test ---------------------- + + def test_encode_hello_round_trips(self) -> None: + line = encode_hello(name="echo", tools=[ECHO_TOOL]) + + assert line.endswith("\n") + assert line.count("\n") == 1 + + frame = decode(line) + + assert frame["type"] == "hello" + assert frame["protocol"] == 1 + assert frame["name"] == "echo" + assert frame["tools"] == [ECHO_TOOL] + + def test_encode_invoke_round_trips_call_id_under_wire_key_id(self) -> None: + line = encode_invoke(call_id="call-1", name="echo", args={"text": "ping"}) + + assert line.endswith("\n") + assert line.count("\n") == 1 + + frame = decode(line) + + assert frame["type"] == "invoke" + assert frame["protocol"] == 1 + assert frame["id"] == "call-1" + assert "call_id" not in frame + assert frame["name"] == "echo" + assert frame["args"] == {"text": "ping"} + + def test_encode_result_round_trips_call_id_under_wire_key_id(self) -> None: + line = encode_result(call_id="call-2", value={"text": "ping"}) + + assert line.endswith("\n") + assert line.count("\n") == 1 + + frame = decode(line) + + assert frame["type"] == "result" + assert frame["protocol"] == 1 + assert frame["id"] == "call-2" + assert "call_id" not in frame + assert frame["value"] == {"text": "ping"} + + def test_encode_error_round_trips_call_id_under_wire_key_id(self) -> None: + line = encode_error(call_id="call-3", error="ValueError: boom") + + assert line.endswith("\n") + assert line.count("\n") == 1 + + frame = decode(line) + + assert frame["type"] == "error" + assert frame["protocol"] == 1 + assert frame["id"] == "call-3" + assert "call_id" not in frame + assert frame["error"] == "ValueError: boom" + + def test_encode_shutdown_round_trips(self) -> None: + line = encode_shutdown() + + assert line.endswith("\n") + assert line.count("\n") == 1 + + frame = decode(line) + + assert frame["type"] == "shutdown" + assert frame["protocol"] == 1 + + def test_encode_shutdown_carries_no_payload(self) -> None: + frame = decode(encode_shutdown()) + + assert set(frame) == {"type", "protocol"} + + # --- Edge: decode rejections -------------------------------------- + + def test_decode_rejects_protocol_zero_hello(self) -> None: + with pytest.raises(ProtocolError) as excinfo: + decode(PROTOCOL_ZERO_HELLO) + + assert "protocol" in str(excinfo.value) + + @pytest.mark.parametrize( + "line", + [ + pytest.param("not json at all", id="non-json"), + pytest.param('["hello", 1]', id="json-array"), + pytest.param('{"protocol": 1, "name": "echo"}', id="missing-type"), + pytest.param('{"type": "bogus", "protocol": 1}', id="unknown-type"), + pytest.param('{"type": "hello", "name": "echo"}', id="missing-protocol"), + ], + ) + def test_decode_rejects_malformed_frames(self, line: str) -> None: + with pytest.raises(ProtocolError): + decode(line) + + # --- Basics: signature facts -------------------------------------- + + def test_signature_facts_returns_a_list_not_a_json_schema(self) -> None: + facts = _sample_facts() + + assert isinstance(facts, list) + for fact in facts: + assert isinstance(fact, dict) + assert JSON_SCHEMA_KEYS.isdisjoint(fact) + assert set(fact) <= FACT_KEYS + + def test_signature_facts_pins_every_parameter(self) -> None: + assert _sample_facts() == [ + { + "name": "text", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "str", + "has_default": False, + }, + { + "name": "count", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "int", + "has_default": True, + "default": 3, + }, + { + "name": "flag", + "kind": "KEYWORD_ONLY", + "annotation": "bool", + "has_default": True, + "default": False, + }, + ] + + def test_signature_facts_omits_default_key_without_a_default(self) -> None: + text_fact = _sample_facts()[0] + + assert text_fact["has_default"] is False + assert "default" not in text_fact + + def test_signature_facts_keeps_defaults_typed(self) -> None: + _, count_fact, flag_fact = _sample_facts() + + assert count_fact["has_default"] is True + assert count_fact["default"] == 3 + assert flag_fact["has_default"] is True + assert flag_fact["default"] is False + + # --- Basics: rebuild_signature is the inverse --------------------- + + def test_rebuild_signature_restores_names_kinds_and_defaults(self) -> None: + signature = rebuild_signature(_sample_facts()) + parameters = signature.parameters + + assert list(parameters) == ["text", "count", "flag"] + assert parameters["text"].kind is inspect.Parameter.POSITIONAL_OR_KEYWORD + assert parameters["count"].kind is inspect.Parameter.POSITIONAL_OR_KEYWORD + assert parameters["flag"].kind is inspect.Parameter.KEYWORD_ONLY + assert parameters["text"].default is inspect.Parameter.empty + assert parameters["count"].default == 3 + assert parameters["flag"].default is False + + def test_rebuild_signature_maps_builtin_annotations_to_type_objects(self) -> None: + parameters = rebuild_signature(_sample_facts()).parameters + + assert parameters["text"].annotation is str + assert parameters["count"].annotation is int + assert parameters["flag"].annotation is bool + + def test_rebuild_signature_keeps_unknown_annotations_as_strings(self) -> None: + signature = rebuild_signature( + [ + { + "name": "thing", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "MyThing", + "has_default": False, + } + ] + ) + + assert signature.parameters["thing"].annotation == "MyThing" + + def test_rebuild_signature_maps_empty_annotation_to_parameter_empty(self) -> None: + signature = rebuild_signature( + [ + { + "name": "raw", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "", + "has_default": False, + } + ] + ) + + assert signature.parameters["raw"].annotation is inspect.Parameter.empty diff --git a/tests/test_provider_worker/test_proxy.py b/tests/test_provider_worker/test_proxy.py new file mode 100644 index 0000000..13066fb --- /dev/null +++ b/tests/test_provider_worker/test_proxy.py @@ -0,0 +1,118 @@ +"""Proxy — hello signature facts become published FastMCP tools. + +``bind_tools`` is the only place a worker plane's catalog turns into MCP +metadata. The hello frame here is written by hand (the shape the child +promises), and the callable it produces is bound to a real ``FastMCP``: the +JSON Schema in the assertions is FastMCP's, produced from the rebuilt +signature, never hand-rolled by the proxy. + +The child is a fake ``invoke`` that records its calls, so nothing in this file +touches a subprocess. +""" + +from __future__ import annotations + +from fastmcp import FastMCP +from fastmcp.tools import Tool + +from molmcp.provider_worker import proxy + +#: The input schema FastMCP publishes for ``echo(text: str)``. Hard-coded: +#: the proxy is correct when FastMCP sees the same signature the child sent. +_ECHO_INPUT_SCHEMA: dict[str, object] = { + "additionalProperties": False, + "properties": {"text": {"type": "string"}}, + "required": ["text"], + "type": "object", +} + + +def _hello() -> dict[str, object]: + """One hello frame, built by hand, carrying signature facts only.""" + return { + "type": "hello", + "protocol": 1, + "name": "echo", + "tools": [ + { + "name": "echo", + "attribute": "echo", + "doc": "Echo text back.", + "annotations": { + "read_only_hint": True, + "destructive_hint": False, + "idempotent_hint": True, + "open_world_hint": False, + }, + "parameters": [ + { + "name": "text", + "kind": "POSITIONAL_OR_KEYWORD", + "annotation": "str", + "has_default": False, + } + ], + } + ], + } + + +class _RecordingInvoke: + """The Supervisor seam: records ``(name, args)`` and answers like echo.""" + + def __init__(self) -> None: + self.calls: list[tuple[str, dict[str, object]]] = [] + + def __call__(self, name: str, args: dict[str, object]) -> dict[str, object]: + self.calls.append((name, dict(args))) + return {"text": args["text"]} + + +async def _published(mcp: FastMCP, name: str) -> Tool: + """The one published tool called *name*.""" + by_name = {tool.name: tool for tool in await mcp.list_tools()} + assert name in by_name, f"{name!r} not published: {sorted(by_name)}" + return by_name[name] + + +class TestProxy: + """One ``bind_tools`` concern per test, against a real FastMCP.""" + + async def test_bind_tools_publishes_the_bare_name(self) -> None: + mcp = FastMCP("echo") + + bound = proxy.bind_tools(mcp, _hello(), _RecordingInvoke()) + + assert bound == ["echo"] + assert {tool.name for tool in await mcp.list_tools()} == {"echo"} + + async def test_description_and_schema_come_from_the_hello_facts(self) -> None: + mcp = FastMCP("echo") + proxy.bind_tools(mcp, _hello(), _RecordingInvoke()) + + tool = await _published(mcp, "echo") + + assert tool.description == "Echo text back." + assert tool.parameters == _ECHO_INPUT_SCHEMA + + async def test_annotations_survive_the_wire(self) -> None: + mcp = FastMCP("echo") + proxy.bind_tools(mcp, _hello(), _RecordingInvoke()) + + annotations = (await _published(mcp, "echo")).annotations + + assert annotations is not None + assert annotations.read_only_hint is True + assert annotations.destructive_hint is False + assert annotations.idempotent_hint is True + assert annotations.open_world_hint is False + + async def test_calling_the_tool_routes_through_invoke(self) -> None: + mcp = FastMCP("echo") + invoke = _RecordingInvoke() + proxy.bind_tools(mcp, _hello(), invoke) + + result = await mcp.call_tool("echo", {"text": "ping"}) + + assert invoke.calls == [("echo", {"text": "ping"})] + assert result.structured_content == {"text": "ping"} diff --git a/tests/test_provider_worker/test_supervisor.py b/tests/test_provider_worker/test_supervisor.py new file mode 100644 index 0000000..75f3716 --- /dev/null +++ b/tests/test_provider_worker/test_supervisor.py @@ -0,0 +1,277 @@ +"""Supervisor — the one owner of the child process, driven through a fake spawn. + +Every test injects ``spawn=``: no real ``subprocess.Popen`` is created here, so +the module is proved in isolation from ``child.py``. The fake process is a +Popen stand-in — text ``stdin`` / ``stdout`` plus ``wait`` / ``terminate`` / +``poll`` — and it records what the Supervisor did to it. + +The wire lines are hard-coded duplex v1 text rather than ``protocol`` encoder +output: a Supervisor that agrees with a broken encoder is still broken. +""" + +from __future__ import annotations + +import ast +import io +import json +import subprocess +import sys +from collections.abc import Callable +from pathlib import Path + +import pytest +from _ast_checks import reads_environment + +from molmcp.provider_worker import supervisor as supervisor_module + +_FIXTURES = Path(__file__).parent / "fixtures" +_ENTRYPOINT = "echo:EchoProvider" + +#: A duplex v1 greeting, written out by hand. +_HELLO_LINE = '{"type": "hello", "protocol": 1, "name": "echo", "tools": []}\n' + +#: The same greeting from a child speaking a protocol this parent does not +#: know. Version mismatch is a hard failure, never a silent downgrade. +_STALE_HELLO_LINE = '{"type": "hello", "protocol": 0, "name": "echo", "tools": []}\n' + +#: A queued child line: fixed text, or a callable resolved when it is read +#: (so a reply can echo back the call id the Supervisor just wrote). +Reply = str | Callable[[], str] + + +class _RecordingStdin(io.StringIO): + """Child stdin that keeps every chunk written to it, even once closed.""" + + def __init__(self) -> None: + super().__init__() + self.writes: list[str] = [] + self.flushes = 0 + + def write(self, s: str) -> int: + written = super().write(s) + self.writes.append(s) + return written + + def flush(self) -> None: + super().flush() + self.flushes += 1 + + @property + def lines(self) -> list[str]: + """Every complete NDJSON line the Supervisor sent, in order.""" + return "".join(self.writes).splitlines() + + +class _ReplyStream: + """Child stdout: one queued line per ``readline``, then EOF.""" + + def __init__(self, replies: list[Reply]) -> None: + self._replies: list[Reply] = list(replies) + + def queue(self, reply: Reply) -> None: + """Make one more line available to the next ``readline``.""" + self._replies.append(reply) + + def readline(self) -> str: + if not self._replies: + return "" + reply = self._replies.pop(0) + return reply() if callable(reply) else reply + + def close(self) -> None: + self._replies.clear() + + +class _FakeProcess: + """A ``Popen`` stand-in that records its own lifecycle calls.""" + + def __init__(self, replies: list[Reply], *, wait_times_out: bool = False) -> None: + self.stdin = _RecordingStdin() + self.stdout = _ReplyStream(replies) + self.wait_calls: list[float | None] = [] + self.terminate_calls = 0 + self.wait_times_out = wait_times_out + self.returncode: int | None = None + + def queue(self, reply: Reply) -> None: + """Queue one more line for the Supervisor to read.""" + self.stdout.queue(reply) + + def wait(self, timeout: float | None = None) -> int: + self.wait_calls.append(timeout) + # A terminated child is reaped; only the first wait can hang. + if self.wait_times_out and not self.terminate_calls: + raise subprocess.TimeoutExpired(cmd="child.py", timeout=timeout or 0.0) + self.returncode = 0 + return 0 + + def terminate(self) -> None: + self.terminate_calls += 1 + self.returncode = -15 + + def poll(self) -> int | None: + return self.returncode + + +class _SpawnRecorder: + """The ``spawn`` seam: records each argv, hands back one prepared process.""" + + def __init__(self, process: _FakeProcess) -> None: + self.process = process + self.argvs: list[list[str]] = [] + + def __call__(self, argv: list[str]) -> _FakeProcess: + self.argvs.append(list(argv)) + return self.process + + +def _sent(process: _FakeProcess) -> list[dict[str, object]]: + """Every frame the Supervisor wrote to the child, decoded.""" + return [json.loads(line) for line in process.stdin.lines] + + +def _sent_of_type(process: _FakeProcess, kind: str) -> list[dict[str, object]]: + return [frame for frame in _sent(process) if frame.get("type") == kind] + + +def _answer(process: _FakeProcess, **payload: object) -> str: + """A child reply to the frame just written, echoing its call id back.""" + last = _sent(process)[-1] + return json.dumps({"protocol": 1, "id": last["id"], **payload}) + "\n" + + +class TestSupervisor: + """One Supervisor concern per test; the child is always a fake.""" + + def test_argv_is_a_path_launch_of_the_child_script(self) -> None: + spawn = _SpawnRecorder(_FakeProcess([_HELLO_LINE])) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, path=_FIXTURES, spawn=spawn + ) + + assert supervisor.argv == [ + sys.executable, + "-P", + str(supervisor_module.CHILD_SCRIPT), + "--entrypoint", + _ENTRYPOINT, + "--path", + str(_FIXTURES), + ] + assert "-m" not in supervisor.argv + # Reading argv must not start anything. + assert spawn.argvs == [] + + def test_child_script_is_a_file_on_disk(self) -> None: + assert supervisor_module.CHILD_SCRIPT.name == "child.py" + assert supervisor_module.CHILD_SCRIPT.is_file() + + def test_start_returns_the_decoded_hello_frame(self) -> None: + process = _FakeProcess([_HELLO_LINE]) + spawn = _SpawnRecorder(process) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, path=_FIXTURES, spawn=spawn + ) + + hello = supervisor.start() + + assert spawn.argvs == [supervisor.argv] + assert hello == { + "type": "hello", + "protocol": 1, + "name": "echo", + "tools": [], + } + + def test_invoke_writes_one_frame_and_returns_the_result_value(self) -> None: + process = _FakeProcess([_HELLO_LINE]) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, + path=_FIXTURES, + spawn=_SpawnRecorder(process), + ) + supervisor.start() + process.queue( + lambda: _answer(process, type="result", ok=True, value={"text": "ping"}) + ) + + value = supervisor.invoke("echo", {"text": "ping"}) + + assert value == {"text": "ping"} + invokes = _sent_of_type(process, "invoke") + assert len(invokes) == 1 + assert invokes[0]["protocol"] == 1 + assert invokes[0]["name"] == "echo" + assert invokes[0]["args"] == {"text": "ping"} + assert invokes[0]["id"] + assert process.stdin.writes[-1].endswith("\n") + assert process.stdin.flushes >= 1 + + def test_invoke_raises_runtime_error_carrying_the_error_text(self) -> None: + process = _FakeProcess([_HELLO_LINE]) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, + path=_FIXTURES, + spawn=_SpawnRecorder(process), + ) + supervisor.start() + process.queue( + lambda: _answer( + process, type="error", ok=False, error="ValueError: no text" + ) + ) + + with pytest.raises(RuntimeError, match="ValueError: no text"): + supervisor.invoke("echo", {}) + + def test_shutdown_is_idempotent(self) -> None: + process = _FakeProcess([_HELLO_LINE]) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, + path=_FIXTURES, + spawn=_SpawnRecorder(process), + ) + supervisor.start() + + supervisor.shutdown() + supervisor.shutdown() + + assert len(_sent_of_type(process, "shutdown")) <= 1 + assert process.wait_calls + assert process.terminate_calls == 0 + assert process.poll() is not None + + def test_shutdown_terminates_a_child_that_will_not_exit(self) -> None: + process = _FakeProcess([_HELLO_LINE], wait_times_out=True) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, + path=_FIXTURES, + spawn=_SpawnRecorder(process), + ) + supervisor.start() + + supervisor.shutdown() + + assert process.wait_calls + assert process.terminate_calls >= 1 + + def test_a_stale_protocol_hello_reaps_the_child_and_raises(self) -> None: + process = _FakeProcess([_STALE_HELLO_LINE]) + supervisor = supervisor_module.Supervisor( + entrypoint=_ENTRYPOINT, + path=_FIXTURES, + spawn=_SpawnRecorder(process), + ) + + with pytest.raises(RuntimeError): + supervisor.start() + + reaped = bool(_sent_of_type(process, "shutdown")) or ( + process.terminate_calls > 0 + ) + assert reaped, "start() must shut the child down before it raises" + + def test_supervisor_never_reads_the_environment(self) -> None: + source = Path(supervisor_module.__file__).read_text(encoding="utf-8") + + assert not reads_environment(ast.parse(source)) diff --git a/tests/test_provider_worker/test_worker.py b/tests/test_provider_worker/test_worker.py new file mode 100644 index 0000000..8591a35 --- /dev/null +++ b/tests/test_provider_worker/test_worker.py @@ -0,0 +1,239 @@ +"""WorkerProvider — the adapter that owns a child plane's whole lifetime. + +``register`` is the only place the two halves meet: a Supervisor starts the +child, the proxy publishes its bare tool names, and only then does the adapter +take over teardown by swapping FastMCP's private ``_lifespan``. FastMCP 4 does +have a public ``mcp.lifespan`` — the inherited ``AggregateProvider.lifespan``, +which takes no server argument and combines the *mounted providers'* lifespans. +That is a different object from the ``FastMCP(lifespan=...)`` callable held in +``_lifespan``, so nothing here reads it; these tests enter +``mcp._lifespan_manager()`` instead. + +The primary reaper is that swapped lifespan: entering and leaving +``mcp._lifespan_manager()`` must leave no child behind, with no ``shutdown()`` +call from the test. ``shutdown()`` is the *explicit abort* — the failure path +and the last resort, never the thing that proves teardown works. + +``create_plane`` is deliberately absent: these are unit tests of the adapter, +and whole-server assembly is a different question from whether this class +starts and reaps a child. +""" + +from __future__ import annotations + +import ast +from pathlib import Path +from typing import Protocol, runtime_checkable + +import pytest +from fastmcp import FastMCP + +import molmcp +from molmcp.provider_worker import WorkerProvider, supervisor, worker + +_FIXTURES = Path(__file__).parent / "fixtures" +_ENTRYPOINT = "echo:EchoProvider" + +_WORKER_FILE = Path(worker.__file__) +_WORKER_SOURCE = _WORKER_FILE.read_text(encoding="utf-8") +_PACKAGE_DIR = _WORKER_FILE.parent + + +@runtime_checkable +class _Reapable(Protocol): + """The one part of ``Popen`` these tests need: is the child still alive?""" + + def poll(self) -> int | None: ... + + +def _members(value: object) -> list[object]: + return list(vars(value).values()) if hasattr(value, "__dict__") else [] + + +def _child_process(provider: WorkerProvider) -> _Reapable: + """The live child, reached through the provider's own attributes. + + The private names are not frozen by the contract, so look for the object + that answers ``poll`` — the process the Supervisor spawned. + """ + for holder in _members(provider): + if isinstance(holder, _Reapable): + return holder + for nested in _members(holder): + if isinstance(nested, _Reapable): + return nested + raise AssertionError("no child process is reachable from the provider") + + +def _install_supervisor(monkeypatch: pytest.MonkeyPatch, factory: type) -> None: + """Swap the Supervisor ``register`` builds, whichever import style it used.""" + monkeypatch.setattr(supervisor, "Supervisor", factory) + monkeypatch.setattr(worker, "Supervisor", factory, raising=False) + + +def _recovery_node_ids(tree: ast.AST) -> set[int]: + """Ids of every node inside an ``except`` handler or a ``finally`` block.""" + ids: set[int] = set() + for node in ast.walk(tree): + if isinstance(node, ast.ExceptHandler): + ids.update(id(child) for child in ast.walk(node)) + elif isinstance(node, ast.Try): + for statement in node.finalbody: + ids.update(id(child) for child in ast.walk(statement)) + return ids + + +class TestWorkerProvider: + """One WorkerProvider concern per test; no ``create_plane`` anywhere.""" + + def test_probe_is_false_when_the_path_is_not_a_directory( + self, tmp_path: Path + ) -> None: + missing = tmp_path / "not-a-checkout" + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=missing) + + assert provider.probe() is False + + def test_register_without_a_checkout_names_what_is_missing( + self, tmp_path: Path + ) -> None: + missing = tmp_path / "not-a-checkout" + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=missing) + + with pytest.raises(RuntimeError) as excinfo: + provider.register(FastMCP("echo")) + + message = str(excinfo.value) + assert str(missing) in message + assert _ENTRYPOINT in message + # A missing checkout is not a missing wheel; do not send anyone to pip. + assert "pip install" not in message + + def test_a_name_outside_the_provider_pattern_is_rejected(self) -> None: + with pytest.raises(ValueError): + WorkerProvider(name="Echo_1", entrypoint=_ENTRYPOINT, path=_FIXTURES) + + async def test_register_publishes_the_bare_tool_name(self) -> None: + mcp = FastMCP("echo") + before = mcp._lifespan + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=_FIXTURES) + + try: + provider.register(mcp) + + assert {tool.name for tool in await mcp.list_tools()} == {"echo"} + # Teardown is now the adapter's; the swap is how it gets there. + assert mcp._lifespan is not before + finally: + provider.shutdown() + + async def test_leaving_the_lifespan_reaps_the_child(self) -> None: + mcp = FastMCP("echo") + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=_FIXTURES) + provider.register(mcp) + process = _child_process(provider) + assert process.poll() is None + + async with mcp._lifespan_manager(): + pass + + # Reaped by the swapped lifespan alone — this test never calls + # shutdown(), because shutdown() is the abort, not the reaper. + assert process.poll() is not None + + def test_shutdown_aborts_the_child_and_is_idempotent(self) -> None: + mcp = FastMCP("echo") + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=_FIXTURES) + provider.register(mcp) + process = _child_process(provider) + + provider.shutdown() + provider.shutdown() + + assert process.poll() is not None + + def test_a_failed_register_reaps_and_leaves_the_lifespan_alone( + self, monkeypatch: pytest.MonkeyPatch + ) -> None: + built: list[str] = [] + reaped: list[str] = [] + + class WrongPlaneSupervisor: + """A child greeting as a plane the adapter was not built for.""" + + def __init__( + self, *, entrypoint: str, path: object, **extra: object + ) -> None: + built.append(entrypoint) + + def start(self) -> dict[str, object]: + return { + "type": "hello", + "protocol": 1, + "name": "other", + "tools": [], + } + + def invoke(self, name: str, args: dict[str, object]) -> object: + raise AssertionError("register must fail before any invoke") + + def shutdown(self) -> None: + reaped.append("shutdown") + + _install_supervisor(monkeypatch, WrongPlaneSupervisor) + mcp = FastMCP("echo") + before = mcp._lifespan + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=_FIXTURES) + + with pytest.raises(ValueError): + provider.register(mcp) + + assert built == [_ENTRYPOINT] + assert reaped == ["shutdown"] + # The swap never happened, so the server owes this adapter nothing. + assert mcp._lifespan is before + + def test_the_adapter_has_no_close_and_no_public_export(self) -> None: + assert not hasattr(WorkerProvider, "close") + assert "WorkerProvider" not in molmcp.__all__ + + def test_the_package_registers_no_atexit_hook(self) -> None: + offenders = [ + path.name + for path in sorted(_PACKAGE_DIR.rglob("*.py")) + if "atexit.register" in path.read_text(encoding="utf-8") + ] + + assert offenders == [] + + def test_one_finalize_and_it_sits_on_the_success_path(self) -> None: + assert _WORKER_SOURCE.count("weakref.finalize(") == 1 + tree = ast.parse(_WORKER_SOURCE) + finalize_calls = [ + node + for node in ast.walk(tree) + if isinstance(node, ast.Call) + and isinstance(node.func, ast.Attribute) + and node.func.attr == "finalize" + ] + assert len(finalize_calls) == 1 + + finalize = finalize_calls[0] + assert id(finalize) not in _recovery_node_ids(tree), ( + "the fallback is for a registered server, not for a failed register" + ) + + swap_lines = [ + node.lineno + for node in ast.walk(tree) + if isinstance(node, ast.Assign) + for target in node.targets + if isinstance(target, ast.Attribute) and target.attr == "_lifespan" + ] + assert swap_lines, "register must swap mcp._lifespan" + assert finalize.lineno > max(swap_lines) + + def test_worker_provider_satisfies_the_provider_protocol(self) -> None: + provider = WorkerProvider(name="echo", entrypoint=_ENTRYPOINT, path=_FIXTURES) + + assert isinstance(provider, molmcp.Provider) diff --git a/tests/test_runtime.py b/tests/test_runtime.py index 3858b1e..8f0d457 100644 --- a/tests/test_runtime.py +++ b/tests/test_runtime.py @@ -1,11 +1,19 @@ from __future__ import annotations +import ast +import inspect import json +import sys +from collections.abc import Iterator from pathlib import Path +import pytest + from molmcp import CollectionIndex, create_plane, runtime +from molmcp.components.models import ComponentKind, ComponentSpec from molmcp.config import AppConfig, load_config from molmcp.discovery.config import DEFAULT_EXCLUDES +from molmcp.discovery.overlay import CapabilityOverlay, OverlayContribution from molmcp.environment import DiscoveredSource, EnvironmentReport @@ -141,3 +149,219 @@ def test_config_summary_includes_secret_free_discovery(tmp_path, monkeypatch): assert "secret" not in lowered assert "password" not in lowered assert "token" not in lowered + + +# --- session capability overlays / build_collection extras ----------------- + +_SESSION_OVERLAYS = "_session_capability_overlays" + +_OVERLAY_FACTORY_MODULE = '''\ +"""Checkout-side overlay module reached through ComponentSpec.entrypoint.""" + + +class DemoOverlay: + name = "demo" + + def applies_to(self, snapshot): + return True + + def contribute(self, graph): + return "demo-contribution" + + +def make_overlay(): + return DemoOverlay() +''' + +_NON_OVERLAY_FACTORY_MODULE = '''\ +"""Checkout-side module whose factory returns a non-overlay object.""" + + +def make_overlay(): + return object() +''' + + +class _FakeOverlay: + """Minimal in-process object satisfying ``CapabilityOverlay``.""" + + def __init__(self, name: str) -> None: + self.name = name + + def applies_to(self, snapshot: object) -> bool: + return True + + def contribute(self, graph: object) -> OverlayContribution: + return OverlayContribution() + + +class _RecordedEngine: + """Stand-in for ``DiscoveryEngine`` that records the overlays argument.""" + + def __init__(self, config: object = None, overlays: object = None) -> None: + self.config = config + self.overlays = overlays + + def query(self, spec: str) -> object: + raise NotImplementedError("the recording engine is never queried") + + +def _record_engines(monkeypatch: pytest.MonkeyPatch) -> list[_RecordedEngine]: + """Capture every engine ``build_collection`` constructs.""" + recorded: list[_RecordedEngine] = [] + + def build(config: object = None, overlays: object = None) -> _RecordedEngine: + engine = _RecordedEngine(config, overlays) + recorded.append(engine) + return engine + + monkeypatch.setattr(runtime, "DiscoveryEngine", build) + return recorded + + +def _bare_config(tmp_path: Path) -> AppConfig: + return AppConfig.from_dict({"schema_version": "2"}, workspace_root=tmp_path) + + +def _write_checkout(tmp_path: Path, module_name: str, source: str) -> Path: + """Write one overlay module into a fake checkout tree; return the tree.""" + tree = tmp_path / "checkout" + overlays = tree / "overlays" + overlays.mkdir(parents=True, exist_ok=True) + (overlays / f"{module_name}.py").write_text(source, encoding="utf-8") + return tree + + +def _overlay_seed(component_name: str, module_name: str) -> ComponentSpec: + return ComponentSpec( + kind=ComponentKind.OVERLAY, + name=component_name, + id=f"overlay.{component_name}", + path=f"overlays/{module_name}.py", + entrypoint=f"{module_name}:make_overlay", + ) + + +def _runtime_source() -> str: + return Path(runtime.__file__).read_text(encoding="utf-8") + + +def _session_overlay_node(source: str) -> ast.FunctionDef: + for node in ast.parse(source).body: + if isinstance(node, ast.FunctionDef) and node.name == _SESSION_OVERLAYS: + return node + raise AssertionError(f"runtime.{_SESSION_OVERLAYS} is not defined") + + +@pytest.fixture +def clean_import_state() -> Iterator[None]: + """Undo any ``sys.path`` / ``sys.modules`` change a checkout import made.""" + saved_path = list(sys.path) + saved_modules = set(sys.modules) + yield + sys.path[:] = saved_path + for name in set(sys.modules) - saved_modules: + del sys.modules[name] + + +def test_build_collection_accepts_extras_defaulting_to_empty_tuple(): + parameters = inspect.signature(runtime.build_collection).parameters + assert "extras" in parameters + default = parameters["extras"].default + assert isinstance(default, tuple) + assert default == () + + +def test_build_collection_empty_extras_equal_load_overlays(tmp_path, monkeypatch): + overlay = _FakeOverlay("from-entry-point") + monkeypatch.setattr(runtime, "load_overlays", lambda: [overlay]) + recorded = _record_engines(monkeypatch) + runtime.build_collection(_bare_config(tmp_path), extras=()) + assert recorded[0].overlays == [overlay] + + +def test_build_collection_concatenates_load_overlays_then_extras(tmp_path, monkeypatch): + first = _FakeOverlay("first") + second = _FakeOverlay("second") + extra = _FakeOverlay("from-checkout") + monkeypatch.setattr(runtime, "load_overlays", lambda: [first, second]) + recorded = _record_engines(monkeypatch) + runtime.build_collection(_bare_config(tmp_path), extras=(extra,)) + assert recorded[0].overlays == [first, second, extra] + + +def test_build_collection_passes_an_empty_overlay_list_not_none(tmp_path, monkeypatch): + monkeypatch.setattr(runtime, "load_overlays", lambda: []) + recorded = _record_engines(monkeypatch) + runtime.build_collection(_bare_config(tmp_path), extras=()) + assert recorded[0].overlays is not None + assert recorded[0].overlays == [] + + +class TestSessionCapabilityOverlays: + """``runtime._session_capability_overlays`` — the checkout overlay loader.""" + + def test_imports_entrypoint_from_checkout_and_returns_the_instance( + self, tmp_path, clean_import_state + ): + tree = _write_checkout(tmp_path, "demo_factory", _OVERLAY_FACTORY_MODULE) + seed = _overlay_seed("demo", "demo_factory") + loaded = list(runtime._session_capability_overlays((seed,), tree)) + assert len(loaded) == 1 + overlay = loaded[0] + assert isinstance(overlay, CapabilityOverlay) + assert type(overlay).__name__ == "DemoOverlay" + assert overlay.name == "demo" + assert overlay.contribute(None) == "demo-contribution" + + def test_factory_result_that_is_not_an_overlay_names_the_component( + self, tmp_path, clean_import_state + ): + tree = _write_checkout(tmp_path, "faulty_factory", _NON_OVERLAY_FACTORY_MODULE) + seed = _overlay_seed("broken-demo", "faulty_factory") + with pytest.raises(ValueError) as excinfo: + list(runtime._session_capability_overlays((seed,), tree)) + assert "broken-demo" in str(excinfo.value) + + def test_never_calls_load_overlays_and_never_globs_the_tree(self): + source = _runtime_source() + node = _session_overlay_node(source) + called = { + child.func.id if isinstance(child.func, ast.Name) else child.func.attr + for child in ast.walk(node) + if isinstance(child, ast.Call) + and isinstance(child.func, (ast.Name, ast.Attribute)) + } + assert "load_overlays" not in called + assert not called & {"glob", "rglob", "iterdir", "walk"} + segment = ast.get_source_segment(source, node) + assert segment is not None + assert "glob" not in segment + + def test_is_private_with_no_public_loader_alias(self): + loader = getattr(runtime, _SESSION_OVERLAYS) + assert loader.__name__.startswith("_") + assert _SESSION_OVERLAYS not in getattr(runtime, "__all__", ()) + aliases = [ + name + for name, value in vars(runtime).items() + if value is loader and not name.startswith("_") + ] + assert aliases == [] + assert not hasattr(runtime, "overlay_loader") + + def test_the_parameters_are_seeds_and_a_base(self): + """``base``, not ``tree_path``: what arrives is a resolved base. + + A harness catalog may declare a ``component_root``, and from that + moment the second argument is ``ComponentFold.root_for``'s answer + rather than the checkout tree ``harness.toml`` sits at. A parameter + still naming it a tree would be a comment that lies about half the + cases, and it is a name the caller may pass by keyword. + + The pin lives here because this class owns the loader's contract. + A ``tests/test_stack.py`` test going red because a runtime parameter + was renamed would be choreography, not the owner's contract. + """ + loader = getattr(runtime, _SESSION_OVERLAYS) + assert tuple(inspect.signature(loader).parameters) == ("seeds", "base") diff --git a/tests/test_settings.py b/tests/test_settings.py index 682268c..83a1fb9 100644 --- a/tests/test_settings.py +++ b/tests/test_settings.py @@ -9,21 +9,17 @@ from __future__ import annotations +import dataclasses +import inspect import json +import pathlib +import sys import pytest from molmcp import settings as st -@pytest.fixture -def home(tmp_path, monkeypatch): - fake = tmp_path / "home" - fake.mkdir() - monkeypatch.setattr(st.Path, "home", staticmethod(lambda: fake)) - return fake - - def _write(path, data) -> None: path.parent.mkdir(parents=True, exist_ok=True) path.write_text(json.dumps(data), encoding="utf-8") @@ -162,3 +158,1098 @@ def test_booleans_and_integers_are_parsed_from_the_command_line(self, home): data = json.loads(st.user_settings_path().read_text()) assert data["indexWorkspace"] is False assert data["maxCacheBytes"] == 1048576 + + +class TestHarnessSourceEdit: + """The verbs that address one ``harness`` entry by origin, not by name. + + ``harness`` is a list of objects, so the string-valued verbs one class + below refuse it outright; these are what authors an entry instead of an + editor. They do not retire the editor: they write into a file that + already parses, so one that fails validation on read still needs one. + + The address is the locator's ``origin_key``. A second spelling of the + same GitHub repository updates that one entry in place; a new origin is + appended **last**. The alias is optional: the first insert without one + is named ``origin``, and a later insert without one is refused once that + alias is taken. ``enable`` / ``disable`` empty means "leave as it was" + — on insert that is ``None`` (all), which the file records by omitting + the key. ``disable=("all",)`` stores ``[]`` and keeps the source. + + Two orderings are binding rather than incidental. Arguments are + validated by constructing a :class:`~molmcp.settings.HarnessSource` + *before* the file is read, so a refused call leaves no file behind at + all; and dropping the last entry leaves ``"harness": []`` rather than + removing the key, which is ``remove_value``'s different job. + """ + + def test_set_harness_source_takes_a_locator_and_keyword_only_edits(self): + parameters = inspect.signature(st.set_harness_source).parameters + + assert list(parameters) == ["path", "locator", "alias", "enable", "disable"] + assert parameters["locator"].kind is inspect.Parameter.POSITIONAL_OR_KEYWORD + assert parameters["alias"].kind is inspect.Parameter.KEYWORD_ONLY + + def test_the_default_alias_constant_is_origin(self): + assert st.DEFAULT_HARNESS_ALIAS == "origin" + + def test_a_first_insert_without_alias_is_named_origin_and_omits_enable( + self, home, tmp_path + ): + path = st.user_settings_path() + + st.set_harness_source(path, "molcrafts/harness") + + assert json.loads(path.read_text()) == { + "harness": [{"name": "origin", "locator": "molcrafts/harness"}] + } + loaded = st.load_settings(tmp_path / "repo").harness + assert loaded == (st.HarnessSource(name="origin", locator="molcrafts/harness"),) + assert loaded[0].enable is None + assert "enable" not in json.loads(path.read_text())["harness"][0] + + def test_the_same_origin_under_a_new_spelling_updates_that_entry_in_place( + self, home + ): + path = st.user_settings_path() + st.set_harness_source(path, "MolCrafts/harness", alias="official") + + st.set_harness_source(path, "https://github.com/MolCrafts/harness.git") + + entries = json.loads(path.read_text())["harness"] + assert len(entries) == 1 + assert entries[0] == { + "name": "official", + "locator": "https://github.com/MolCrafts/harness.git", + } + + def test_an_update_without_alias_keeps_the_name_already_stored(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", alias="official") + + st.set_harness_source(path, "molcrafts/harness@dev") + + assert json.loads(path.read_text())["harness"][0]["name"] == "official" + assert json.loads(path.read_text())["harness"][0]["locator"] == ( + "molcrafts/harness@dev" + ) + + def test_a_new_origin_with_an_alias_is_appended_last(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + st.set_harness_source(path, "acme/harness", alias="mine") + + entries = json.loads(path.read_text())["harness"] + assert [entry["name"] for entry in entries] == ["origin", "mine"] + assert [entry["locator"] for entry in entries] == [ + "molcrafts/harness", + "acme/harness", + ] + + def test_a_second_origin_without_alias_is_refused_once_origin_is_taken(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + before = path.read_text(encoding="utf-8") + + with pytest.raises(st.SettingsError) as excinfo: + st.set_harness_source(path, "acme/harness") + + assert "alias" in str(excinfo.value) + assert path.read_text(encoding="utf-8") == before + + def test_a_second_origin_without_alias_is_named_origin_when_that_alias_is_free( + self, home + ): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", alias="official") + + st.set_harness_source(path, "acme/harness") + + assert [entry["name"] for entry in json.loads(path.read_text())["harness"]] == [ + "official", + "origin", + ] + + def test_an_alias_renames_the_matched_origin(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + st.set_harness_source(path, "molcrafts/harness", alias="official") + + assert json.loads(path.read_text())["harness"] == [ + {"name": "official", "locator": "molcrafts/harness"} + ] + + def test_an_alias_that_another_entry_already_uses_is_refused(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", alias="official") + st.set_harness_source(path, "acme/harness", alias="mine") + before = path.read_text(encoding="utf-8") + + with pytest.raises(st.SettingsError) as excinfo: + st.set_harness_source(path, "acme/harness", alias="official") + + assert "official" in str(excinfo.value) + assert path.read_text(encoding="utf-8") == before + + def test_disable_all_persists_an_empty_enable_list_and_keeps_the_entry( + self, home, tmp_path + ): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + st.set_harness_source(path, "molcrafts/harness", disable=("all",)) + + assert json.loads(path.read_text())["harness"] == [ + {"name": "origin", "locator": "molcrafts/harness", "enable": []} + ] + loaded = st.load_settings(tmp_path / "repo").harness + assert len(loaded) == 1 + assert loaded[0].enable == () + assert loaded[0].name == "origin" + + def test_disable_all_on_insert_still_writes_the_source(self, home, tmp_path): + path = st.user_settings_path() + + st.set_harness_source(path, "molcrafts/harness", disable=("all",)) + + assert json.loads(path.read_text())["harness"][0]["enable"] == [] + assert st.load_settings(tmp_path / "repo").harness[0].enable == () + + def test_named_enable_on_insert_is_the_list_that_lands_in_the_file(self, home): + path = st.user_settings_path() + + st.set_harness_source(path, "molcrafts/harness", enable=("sci", "dev")) + + assert json.loads(path.read_text())["harness"] == [ + { + "name": "origin", + "locator": "molcrafts/harness", + "enable": ["sci", "dev"], + } + ] + + def test_named_enable_replaces_the_all_sentinel(self, home, tmp_path): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + st.set_harness_source(path, "molcrafts/harness", enable=("sci",)) + + assert st.load_settings(tmp_path / "repo").harness[0].enable == ("sci",) + + def test_named_enable_unions_an_already_explicit_list(self, home, tmp_path): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", enable=("sci",)) + + st.set_harness_source(path, "molcrafts/harness", enable=("dev",)) + + assert st.load_settings(tmp_path / "repo").harness[0].enable == ("sci", "dev") + + def test_named_disable_subtracts_from_an_explicit_list(self, home, tmp_path): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", enable=("sci", "dev")) + + st.set_harness_source(path, "molcrafts/harness", disable=("sci",)) + + assert st.load_settings(tmp_path / "repo").harness[0].enable == ("dev",) + + def test_named_disable_of_the_last_name_leaves_the_empty_tuple_not_all( + self, home, tmp_path + ): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", enable=("sci",)) + + st.set_harness_source(path, "molcrafts/harness", disable=("sci",)) + + loaded = st.load_settings(tmp_path / "repo").harness[0] + assert loaded.enable == () + assert json.loads(path.read_text())["harness"][0]["enable"] == [] + + def test_named_disable_on_the_all_sentinel_is_refused(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + before = path.read_text(encoding="utf-8") + + with pytest.raises(st.SettingsError): + st.set_harness_source(path, "molcrafts/harness", disable=("sci",)) + + assert path.read_text(encoding="utf-8") == before + + def test_enable_all_restores_the_omitted_key(self, home, tmp_path): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", enable=("sci",)) + + st.set_harness_source(path, "molcrafts/harness", enable=("all",)) + + assert "enable" not in json.loads(path.read_text())["harness"][0] + assert st.load_settings(tmp_path / "repo").harness[0].enable is None + + def test_enable_all_must_not_share_the_call_with_a_named_enable(self, home): + with pytest.raises(st.SettingsError): + st.set_harness_source( + st.user_settings_path(), + "molcrafts/harness", + enable=("all", "sci"), + ) + + assert not st.user_settings_path().exists() + + def test_enable_all_must_not_share_the_call_with_disable_all(self, home): + with pytest.raises(st.SettingsError): + st.set_harness_source( + st.user_settings_path(), + "molcrafts/harness", + enable=("all",), + disable=("all",), + ) + + assert not st.user_settings_path().exists() + + def test_empty_enable_and_disable_on_update_leave_the_field_as_it_was( + self, home, tmp_path + ): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", enable=("sci",)) + + st.set_harness_source(path, "molcrafts/harness") + + assert st.load_settings(tmp_path / "repo").harness[0].enable == ("sci",) + + def test_a_refused_locator_creates_no_file_at_all(self, home): + with pytest.raises(st.SettingsError): + st.set_harness_source(st.user_settings_path(), "./checkout") + + assert not st.user_settings_path().exists() + + def test_a_refused_enable_token_creates_no_file_at_all(self, home): + with pytest.raises(st.SettingsError): + st.set_harness_source( + st.user_settings_path(), "molcrafts/harness", enable=("foo_bar",) + ) + + assert not st.user_settings_path().exists() + + def test_the_dataclass_message_is_the_one_the_operator_reads(self, home): + with pytest.raises(ValueError) as from_the_type: + st.HarnessSource(name="my harness", locator="molcrafts/harness") + + with pytest.raises(st.SettingsError) as from_the_verb: + st.set_harness_source( + st.user_settings_path(), "molcrafts/harness", alias="my harness" + ) + + assert str(from_the_type.value) in str(from_the_verb.value) + + def test_match_harness_source_is_exported_beside_the_edit_verbs(self): + assert callable(st.match_harness_source) + assert "match_harness_source" in st.__all__ + + def test_match_harness_source_hits_an_exact_name_first(self): + sources = ( + st.HarnessSource(name="official", locator="molcrafts/harness"), + st.HarnessSource(name="mine", locator="acme/harness"), + ) + + assert st.match_harness_source(sources, "mine") == sources[1] + + def test_match_harness_source_hits_origin_key_when_the_token_is_not_a_name(self): + sources = (st.HarnessSource(name="official", locator="molcrafts/harness"),) + + matched = st.match_harness_source( + sources, "https://github.com/MolCrafts/harness.git" + ) + + assert matched == sources[0] + + def test_match_harness_source_treats_a_ref_as_not_part_of_identity(self): + sources = (st.HarnessSource(name="official", locator="molcrafts/harness"),) + + assert st.match_harness_source(sources, "MolCrafts/harness@dev") == sources[0] + + def test_match_harness_source_prefers_name_when_a_token_could_be_either(self): + sources = ( + st.HarnessSource(name="molcrafts/harness", locator="acme/other"), + st.HarnessSource(name="official", locator="molcrafts/harness"), + ) + + assert st.match_harness_source(sources, "molcrafts/harness") == sources[0] + + def test_remove_drops_the_named_entry_and_leaves_the_others_in_order(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "acme/first", alias="first") + st.set_harness_source(path, "acme/second", alias="second") + st.set_harness_source(path, "acme/third", alias="third") + + st.remove_harness_source(path, "second") + + entries = json.loads(path.read_text())["harness"] + assert [entry["name"] for entry in entries] == ["first", "third"] + + def test_remove_accepts_a_locator_for_the_same_origin(self, home, tmp_path): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness", alias="official") + + st.remove_harness_source(path, "https://github.com/molcrafts/harness") + + assert json.loads(path.read_text())["harness"] == [] + assert st.load_settings(tmp_path / "repo").harness == () + + def test_removing_the_last_entry_leaves_an_empty_list_not_a_missing_key( + self, home, tmp_path + ): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + st.remove_harness_source(path, "origin") + + assert json.loads(path.read_text())["harness"] == [] + assert st.load_settings(tmp_path / "repo").harness == () + + def test_removing_an_absent_token_reports_that_token(self, home): + path = st.user_settings_path() + st.set_harness_source(path, "molcrafts/harness") + + with pytest.raises(st.SettingsError) as excinfo: + st.remove_harness_source(path, "official") + + assert "official" in str(excinfo.value) + + def test_removing_from_a_file_with_no_harness_key_reports_the_file(self, home): + path = st.user_settings_path() + _write(path, {"indexWorkspace": True}) + + with pytest.raises(st.SettingsError) as excinfo: + st.remove_harness_source(path, "mine") + + assert str(path) in str(excinfo.value) + + @pytest.mark.parametrize("retired", ["owner", "repo", "ref", "path"]) + def test_old_coordinate_keys_are_a_hard_cut(self, home, tmp_path, retired): + _write( + st.user_settings_path(), + {"harness": [{"name": "official", retired: "MolCrafts"}]}, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "molmcp config harness set " in str(excinfo.value) + + def test_a_retired_key_is_a_hard_cut_even_when_locator_is_also_present( + self, home, tmp_path + ): + _write( + st.user_settings_path(), + { + "harness": [ + { + "name": "official", + "locator": "molcrafts/harness", + "owner": "MolCrafts", + } + ] + }, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "molmcp config harness set " in str(excinfo.value) + + def test_an_entry_without_a_locator_is_refused(self, home, tmp_path): + _write( + st.user_settings_path(), + {"harness": [{"name": "official"}]}, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "locator" in str(excinfo.value) + + def test_both_verbs_join_all_beside_the_verb_they_extend(self): + assert ( + st.__all__.index("remove_harness_source") + == st.__all__.index("remove_value") - 1 + ) + assert ( + st.__all__.index("set_harness_source") == st.__all__.index("set_value") - 1 + ) + + +class TestGetValueWalk: + """The dotted read, whose one condition was doing the work of two. + + A key the data does not carry and a path *through* something that is + not an object are different answers. An undeclared key is ``null``, + which reads as "unset"; ``harness.owner`` is not a path at all now that + ``harness`` is a list of named entries, and answering ``null`` there + tells an operator the coordinate is unset rather than unreachable — + the wrong of the two errors, and the one that sends them looking for a + verb to set it with. + + Which case each arm actually serves is easy to get backwards. + ``Settings.to_dict()`` always carries every key, ``cacheDir`` among + them, so a bare ``cacheDir`` read answers ``None`` because the *value* + is ``None`` and the walk ends — never through the missing-key arm at + all. That arm is reachable only for keys ``to_dict()`` does not carry: + ``nope`` and ``sources.nope``. Both are pinned below, because they are + what keeps this fix narrow. + + The head key is deliberately not checked against ``_SCHEMA``: + ``to_dict()`` emits ``layers``, which ``_SCHEMA`` does not declare, so + validating there would break a read that works today. + """ + + @pytest.mark.parametrize( + "key", + ["harness.owner", "cacheDir.x", "excludes.x", "indexWorkspace.x", "layers.x"], + ) + def test_descending_through_a_non_object_names_the_key_it_cannot_walk(self, key): + with pytest.raises(st.SettingsError) as excinfo: + st.get_value(st.Settings().to_dict(), key) + + assert key in str(excinfo.value) + + def test_an_undeclared_top_level_key_still_reads_as_null(self): + data = st.Settings().to_dict() + + assert "nope" not in data + assert st.get_value(data, "nope") is None + + def test_an_undeclared_member_of_a_dict_setting_still_reads_as_null(self): + data = st.Settings().to_dict() + + assert "nope" not in data["sources"] + assert st.get_value(data, "sources.nope") is None + + def test_an_unset_value_reads_as_null_by_the_other_route_entirely(self): + data = st.Settings().to_dict() + + assert "cacheDir" in data + assert st.get_value(data, "cacheDir") is None + + def test_a_key_the_schema_does_not_declare_is_read_rather_than_validated( + self, tmp_path + ): + layer = tmp_path / "settings.json" + data = st.Settings(layers=(layer,)).to_dict() + + assert "layers" not in st._SCHEMA + assert st.get_value(data, "layers") == [str(layer)] + + def test_a_dotted_read_into_a_dict_setting_still_returns_the_member(self): + data = st.Settings(sources={"molpy": "pkg:molpy"}).to_dict() + + assert st.get_value(data, "sources.molpy") == "pkg:molpy" + + +class TestHarnessWriteGuard: + """The string-valued edit verbs cannot author a list of objects. + + ``harness`` became a ``list``, which unlocked two write paths that were + safely refused while it was a ``dict``: ``config set harness x`` parses + to ``["x"]`` and ``config add harness x`` appends the bare string. Both + reach ``write_settings_file`` *before* anything validates, and the + per-entry validator then rejects ``"x"`` on the next read — under + ``load_settings``, hence under ``config list``, ``get``, ``set``, + ``remove`` and ``serve`` alike. ``config harness set`` is no rescue + from that state: it reads through ``read_settings_file`` like every + other verb, so it cannot repair a file it cannot load, and that file + still has to be hand-edited to make the install usable again. The + binding assertions are therefore that the call raises, that **no file + is created**, and that a later ``load_settings`` still works. + + What the guard protects is the line between the two kinds of verb, not + the absence of a writer. ``set`` and ``add`` take a string and still + refuse this key, because a string verb cannot author a list of + objects; the verb that can is ``config harness set``, which addresses + one entry by its ``name`` (``TestHarnessSourceEdit``, above). That is + why the refusals below name a command rather than an editor. + + The refusal is reached through the declared ``_OBJECT_LISTS`` table + rather than a ``"harness"`` literal in either function body, so the next + list of objects closes the same hole by joining the tuple instead of by + someone remembering to add a second branch. + """ + + def test_the_refusal_is_declared_in_a_table_rather_than_branched_on(self): + assert "harness" in st._OBJECT_LISTS + + def test_set_refuses_to_write_a_bare_string_to_the_harness_key(self, home): + with pytest.raises(st.SettingsError): + st.set_value(st.user_settings_path(), "harness", "x") + + assert not st.user_settings_path().exists() + + def test_add_refuses_to_append_a_bare_string_to_the_harness_key(self, home): + with pytest.raises(st.SettingsError): + st.add_value(st.user_settings_path(), "harness", "x") + + assert not st.user_settings_path().exists() + + @pytest.mark.parametrize("write", [st.set_value, st.add_value], ids=["set", "add"]) + def test_a_refused_write_leaves_the_install_loadable(self, home, tmp_path, write): + with pytest.raises(st.SettingsError): + write(st.user_settings_path(), "harness", "x") + + assert st.load_settings(tmp_path / "repo").harness == () + + @pytest.mark.parametrize("member", st._OBJECT_LISTS) + def test_the_refusal_names_the_verb_it_derives_from_the_key(self, home, member): + """The command is built from ``key``, so the table stays truthful. + + A message that hand-wrote ``harness`` would go stale the day a + second list of objects joined :data:`_OBJECT_LISTS`, which is the + drift the table exists to prevent. Naming a verb is only possible + now that one resolves; until this link there was none, which is + why the message pointed at an editor instead. + """ + with pytest.raises(st.SettingsError) as excinfo: + st.set_value(st.user_settings_path(), member, "x") + + assert f"molmcp config {member} set" in str(excinfo.value) + assert "by editing" not in str(excinfo.value) + + def test_the_add_refusal_names_the_set_leaf_the_parser_registers(self, home): + """``config add harness x`` is answered with the leaf that exists. + + There is no ``config harness add``: one entry is authored by name, + and appending is what ``config harness set`` does with a name it + has not seen. Naming an unregistered leaf here would turn this + error message into the next error. + """ + with pytest.raises(st.SettingsError) as excinfo: + st.add_value(st.user_settings_path(), "harness", "x") + + assert "molmcp config harness set" in str(excinfo.value) + assert "by editing" not in str(excinfo.value) + + @pytest.mark.parametrize("member", ["owner", "dev"]) + def test_set_refuses_every_dotted_harness_key_not_only_a_stray_one( + self, home, member + ): + with pytest.raises(st.SettingsError) as excinfo: + st.set_value(st.user_settings_path(), f"harness.{member}", "x") + + assert f"harness.{member}" in str(excinfo.value) + assert "molmcp config harness" in str(excinfo.value) + assert not st.user_settings_path().exists() + + def test_the_dotted_refusal_is_reached_from_remove_as_well_as_set(self, home): + """One sentence serves both leaves, because ``_resolve`` serves both. + + ``_resolve`` is where a dotted key is refused and it cannot see + which verb called it, so its sentence names the ``config harness`` + verbs rather than only ``set``. + """ + with pytest.raises(st.SettingsError) as excinfo: + st.remove_value(st.user_settings_path(), "harness.owner") + + assert "harness.owner" in str(excinfo.value) + assert "molmcp config harness" in str(excinfo.value) + + @pytest.mark.parametrize("key", ["excludes.foo", "cacheDir.x"]) + def test_a_dotted_key_outside_the_table_keeps_the_generic_message(self, home, key): + """Only an ``_OBJECT_LISTS`` head earns the friendlier sentence. + + ``excludes`` and ``cacheDir`` are not lists of entry objects, and + pointing them at a harness verb would be a worse error than the + vague one they get today. + """ + with pytest.raises(st.SettingsError) as excinfo: + st.set_value(st.user_settings_path(), key, "x") + + assert str(excinfo.value) == f"{key!r} is not a settable path" + + def test_remove_refuses_its_value_arm_and_names_the_remove_leaf(self, home): + """A remove is answered with a remove, not with a set. + + ``remove_value``'s list arm compares a string against entry + objects, so ``config remove harness official`` reported that + ``'official'`` was not present while an entry named ``official`` + sat in the file — vague when entries were unnamed, actively false + now that they are named. The guard extends to this arm only: + dropping the whole key is a different operation, pinned by + ``test_remove_still_clears_the_key_and_leaves_a_loadable_file`` + below. Answering a remove with ``config harness set`` would be a + precise misdirection, which is worse than the vague message it + replaces. + """ + path = st.user_settings_path() + _write(path, {"harness": [{"name": "official", "locator": "acme/harness"}]}) + before = path.read_text(encoding="utf-8") + + with pytest.raises(st.SettingsError) as excinfo: + st.remove_value(path, "harness", "official") + + assert "molmcp config harness remove" in str(excinfo.value) + assert "molmcp config harness set" not in str(excinfo.value) + assert "is not present in" not in str(excinfo.value) + assert path.read_text(encoding="utf-8") == before + + def test_remove_still_clears_the_key_and_leaves_a_loadable_file( + self, home, tmp_path + ): + _write( + st.user_settings_path(), + { + "harness": [{"name": "mine", "locator": "acme/harness"}], + "indexWorkspace": True, + }, + ) + + st.remove_value(st.user_settings_path(), "harness") + + assert "harness" not in json.loads(st.user_settings_path().read_text()) + assert st.load_settings(tmp_path / "repo").harness == () + + +class TestHarnessSource: + """One named harness source: three operator fields, identity derived. + + Dataclass fields are exactly ``name``, ``locator``, ``enable``. GitHub + identity and the local path are parsed from ``locator`` at construction + and are not fields — they do not appear in ``asdict`` or in the file. + ``enable`` defaults to ``None`` (all); ``()`` is explicit all-off; a + non-empty tuple is bundle names matching ``COMPONENT_NAME_PATTERN``. + Construction requires a locator: a name-only half-authored entry is no + longer a thing this type can represent. + """ + + def test_fields_are_exactly_name_locator_enable(self): + assert [field.name for field in dataclasses.fields(st.HarnessSource)] == [ + "name", + "locator", + "enable", + ] + + def test_coordinate_fields_are_gone_from_the_type_and_the_module(self): + names = {field.name for field in dataclasses.fields(st.HarnessSource)} + for retired in ("owner", "repo", "ref", "path", "origin_key"): + assert retired not in names + assert not hasattr(st, "HARNESS_COORDINATES") + + def test_enable_defaults_to_none(self): + source = st.HarnessSource(name="official", locator="molcrafts/harness") + + assert source.enable is None + + def test_a_github_locator_keeps_operator_fields_and_derives_identity(self): + source = st.HarnessSource(name="official", locator="MolCrafts/harness@dev") + + assert source.name == "official" + assert source.locator == "MolCrafts/harness@dev" + assert source.enable is None + assert source.origin_key == "molcrafts/harness" + assert source.owner == "molcrafts" + assert source.repo == "harness" + assert source.ref == "dev" + assert source.path == "" + assert source.is_local is False + + def test_asdict_is_only_the_operator_fields(self): + source = st.HarnessSource(name="official", locator="MolCrafts/harness@dev") + + assert dataclasses.asdict(source) == { + "name": "official", + "locator": "MolCrafts/harness@dev", + "enable": None, + } + + def test_a_name_alone_is_not_constructible(self): + with pytest.raises(TypeError): + st.HarnessSource(name="mine") + + @pytest.mark.parametrize("name", ["", " ", "my harness"]) + def test_an_empty_or_whitespace_bearing_name_is_rejected(self, name): + with pytest.raises(ValueError): + st.HarnessSource(name=name, locator="molcrafts/harness") + + def test_a_mixed_case_name_is_as_legal_as_a_mixed_case_source_key(self): + assert ( + st.HarnessSource(name="MolCrafts", locator="molcrafts/harness").name + == "MolCrafts" + ) + assert not hasattr(st, "HARNESS_SOURCE_NAME_PATTERN") + + def test_a_local_locator_derives_the_resolved_path(self, tmp_path): + raw = str(tmp_path / "harness") + source = st.HarnessSource(name="mine", locator=raw) + resolved = str(pathlib.Path(raw).expanduser().resolve()) + + assert source.is_local is True + assert source.path == resolved + assert source.origin_key == resolved + assert source.owner == "" + assert source.repo == "" + assert source.ref == "" + + def test_an_invalid_locator_is_rejected(self): + with pytest.raises(ValueError): + st.HarnessSource(name="mine", locator="./checkout") + + def test_a_locator_that_is_not_a_string_is_refused(self): + with pytest.raises(ValueError): + st.HarnessSource(name="mine", locator=pathlib.Path("/home/me/harness")) + + @pytest.mark.parametrize( + "value", [" ", "/home/me/my harness", "/home/me/harness\t"] + ) + def test_a_locator_carrying_whitespace_is_refused(self, value): + with pytest.raises(ValueError): + st.HarnessSource(name="mine", locator=value) + + def test_a_github_locator_carrying_a_backslash_is_refused(self): + with pytest.raises(ValueError): + st.HarnessSource(name="mine", locator=r"MolCrafts\harness") + + def test_a_windows_drive_locator_is_local_only_on_windows(self): + raw = r"C:\harness" + if sys.platform == "win32": + source = st.HarnessSource(name="mine", locator=raw) + assert source.is_local is True + assert source.locator == raw + else: + with pytest.raises(ValueError): + st.HarnessSource(name="mine", locator=raw) + + def test_enable_empty_tuple_is_stored_as_the_all_off_sentinel(self): + source = st.HarnessSource( + name="official", locator="molcrafts/harness", enable=() + ) + + assert source.enable == () + assert dataclasses.asdict(source)["enable"] == () + + def test_enable_named_tuple_is_stored(self): + source = st.HarnessSource( + name="official", locator="molcrafts/harness", enable=("sci", "dev") + ) + + assert source.enable == ("sci", "dev") + + @pytest.mark.parametrize("token", ["foo_bar", "Sci", "sci_dev"]) + def test_an_enable_name_outside_the_component_pattern_is_refused(self, token): + with pytest.raises(ValueError): + st.HarnessSource( + name="official", locator="molcrafts/harness", enable=(token,) + ) + + def test_https_spelling_shares_origin_key_with_owner_repo(self): + source = st.HarnessSource( + name="official", + locator="https://github.com/MolCrafts/harness.git/", + ) + + assert source.origin_key == "molcrafts/harness" + assert source.owner == "molcrafts" + assert source.repo == "harness" + assert source.ref == "" + + def test_the_instance_is_frozen(self): + source = st.HarnessSource(name="official", locator="molcrafts/harness") + + with pytest.raises(dataclasses.FrozenInstanceError): + source.name = "other" # type: ignore[misc] + + +class TestSettingsHarnessSources: + """``harness`` as a settings key: a list of objects, and no merge channel. + + The list is not merged across layers — the most specific file's list + replaces the others whole — which is the opposite of the ``_MERGED_LISTS`` + members twelve lines above it in the module. The asymmetry is intended: + ``extend`` on a first-wins list would land the user file's entries at the + front and make the user file outrank the project file, the inverse of + every other setting. + """ + + def test_harness_is_a_list_setting_with_no_merge_channel(self): + assert st._SCHEMA.get("harness") is list + assert "harness" not in st._MERGED_DICTS + assert "harness" not in st._MERGED_LISTS + assert "harness" not in st._NESTED_SCHEMA + assert "harness" in st._OBJECT_LISTS + + def test_the_entry_keys_are_derived_from_the_dataclass_fields(self): + assert st._HARNESS_ENTRY_KEYS == { + f.name for f in dataclasses.fields(st.HarnessSource) + } + + def test_operator_fields_are_the_entry_keys_and_identity_is_not(self): + assert st._HARNESS_ENTRY_KEYS == {"name", "locator", "enable"} + for derived in ("origin_key", "ref", "owner", "repo", "path"): + assert derived not in st._HARNESS_ENTRY_KEYS + + def test_two_entries_in_one_file_load_in_file_order(self, home, tmp_path): + _write( + st.user_settings_path(), + { + "harness": [ + {"name": "official", "locator": "molcrafts/harness"}, + {"name": "team", "locator": "acme/harness@v2"}, + ] + }, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert [source.name for source in loaded.harness] == ["official", "team"] + assert [source.origin_key for source in loaded.harness] == [ + "molcrafts/harness", + "acme/harness", + ] + + def test_the_most_specific_layer_replaces_the_list_rather_than_merging( + self, home, tmp_path + ): + _write( + st.user_settings_path(), + {"harness": [{"name": "user", "locator": "user/harness"}]}, + ) + project = tmp_path / "repo" + _write( + st.project_settings_path(project), + {"harness": [{"name": "project", "locator": "project/harness"}]}, + ) + _write( + st.project_settings_path(project, local=True), + {"harness": [{"name": "local", "locator": "local/harness"}]}, + ) + + loaded = st.load_settings(project) + + assert [source.name for source in loaded.harness] == ["local"] + + def test_two_entries_sharing_a_name_in_one_file_are_refused(self, home, tmp_path): + _write( + st.user_settings_path(), + { + "harness": [ + {"name": "twin", "locator": "molcrafts/harness"}, + {"name": "twin", "locator": "acme/harness"}, + ] + }, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "twin" in str(excinfo.value) + + def test_two_entries_sharing_an_origin_key_in_one_file_are_refused( + self, home, tmp_path + ): + _write( + st.user_settings_path(), + { + "harness": [ + {"name": "official", "locator": "MolCrafts/harness"}, + { + "name": "also", + "locator": "https://github.com/molcrafts/harness.git", + }, + ] + }, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "molcrafts/harness" in str(excinfo.value) + + def test_an_omitted_enable_key_loads_as_none(self, home, tmp_path): + _write( + st.user_settings_path(), + {"harness": [{"name": "mine", "locator": "acme/harness"}]}, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert loaded.harness == ( + st.HarnessSource(name="mine", locator="acme/harness"), + ) + assert loaded.harness[0].enable is None + + def test_an_empty_enable_list_loads_as_an_empty_tuple(self, home, tmp_path): + _write( + st.user_settings_path(), + {"harness": [{"name": "mine", "locator": "acme/harness", "enable": []}]}, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert loaded.harness[0].enable == () + + def test_a_named_enable_list_loads_as_a_tuple(self, home, tmp_path): + _write( + st.user_settings_path(), + { + "harness": [ + { + "name": "mine", + "locator": "acme/harness", + "enable": ["sci", "dev"], + } + ] + }, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert loaded.harness[0].enable == ("sci", "dev") + + @pytest.mark.parametrize( + "member", ["dev", "cacheDir", "token", "daily", "telemetry"] + ) + def test_a_stray_entry_member_is_rejected_by_indexed_name( + self, home, tmp_path, member + ): + _write( + st.user_settings_path(), + {"harness": [{"name": "mine", "locator": "acme/harness", member: "x"}]}, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert f"harness[0].{member}" in str(excinfo.value) + + def test_an_entry_without_a_name_is_refused(self, home, tmp_path): + _write( + st.user_settings_path(), + {"harness": [{"locator": "molcrafts/harness"}]}, + ) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "harness[0]" in str(excinfo.value) + assert "name" in str(excinfo.value) + + @pytest.mark.parametrize("table", [{"locator": "molcrafts/harness"}, {}]) + def test_a_harness_table_is_refused_with_the_list_shape( + self, home, tmp_path, table + ): + _write(st.user_settings_path(), {"harness": table}) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "harness" in str(excinfo.value) + assert "list" in str(excinfo.value) + + def test_to_dict_emits_only_the_operator_fields(self): + settings = st.Settings( + harness=(st.HarnessSource(name="official", locator="molcrafts/harness"),) + ) + + assert settings.to_dict()["harness"] == [ + { + "name": "official", + "locator": "molcrafts/harness", + "enable": None, + } + ] + emitted = settings.to_dict()["harness"][0] + for derived in ("origin_key", "ref", "owner", "repo", "path"): + assert derived not in emitted + + def test_a_local_entry_loads_as_written(self, home, tmp_path): + locator = "/opt/harness/mine" + _write( + st.user_settings_path(), + {"harness": [{"name": "mine", "locator": locator}]}, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert loaded.harness == (st.HarnessSource(name="mine", locator=locator),) + assert loaded.harness[0].is_local is True + + def test_a_local_entry_round_trips_through_load_and_to_dict(self, home, tmp_path): + locator = "/opt/harness/mine" + _write( + st.user_settings_path(), + {"harness": [{"name": "mine", "locator": locator}]}, + ) + + loaded = st.load_settings(tmp_path / "repo") + + assert loaded.to_dict()["harness"] == [ + {"name": "mine", "locator": locator, "enable": None} + ] + + def test_a_local_and_a_remote_entry_coexist_in_one_file(self, home, tmp_path): + local = "/opt/harness/mine" + _write( + st.user_settings_path(), + { + "harness": [ + {"name": "official", "locator": "MolCrafts/harness"}, + {"name": "mine", "locator": local}, + ] + }, + ) + + loaded = st.load_settings(tmp_path / "repo") + resolved = str(pathlib.Path(local).expanduser().resolve()) + + assert [(s.name, s.origin_key, s.path) for s in loaded.harness] == [ + ("official", "molcrafts/harness", ""), + ("mine", resolved, resolved), + ] + + def test_an_install_that_names_no_source_has_an_empty_tuple(self): + assert st.Settings().harness == () + + +class TestSettingsHarness: + """The autonomous harness: an ordered list of named sources. + + Each entry is a ``HarnessSource`` — a ``name``, a ``locator``, and an + optional ``enable`` list — and neighbouring settings do not live on + it. A cache location is ``cacheDir`` at the top level, a credential + belongs in the environment rather than a file that can be committed, + and the rest were never molmcp settings at all. + """ + + def test_the_harness_did_not_smuggle_in_neighbouring_settings(self): + for stray in ("shareReceipts", "daily", "telemetry"): + assert stray not in st._SCHEMA + + def test_share_receipts_is_not_a_setting(self, home, tmp_path): + _write(st.user_settings_path(), {"shareReceipts": True}) + + with pytest.raises(st.SettingsError) as excinfo: + st.load_settings(tmp_path / "repo") + + assert "shareReceipts" in str(excinfo.value) + + +class TestNestedSchemaFirstParty: + """First-party planes are named settings, not a generic ``providers`` bag. + + ``molq`` and ``molexp`` each configure one plane, and each knows which + members it reads. A single ``providers`` dict keyed by plane name would + accept any key for any plane: `config set providers.molq.allowsubmit` + would be stored, echoed by `config list`, and read by nothing. The plane + catalog's membership moved to the entry-point group (spec 14); the + settings surface deliberately did not follow it. + """ + + def test_molq_and_molexp_are_dict_valued_first_party_settings(self): + assert st._SCHEMA.get("molq") is dict + assert st._SCHEMA.get("molexp") is dict + + def test_there_is_no_generic_providers_bag(self): + assert "providers" not in st._SCHEMA + assert "providers" not in st._NESTED_SCHEMA + + def test_molq_members_are_exactly_database_and_allow_submit(self): + assert st._NESTED_SCHEMA["molq"] == frozenset({"database", "allowSubmit"}) + + def test_molexp_members_are_exactly_workspace(self): + assert st._NESTED_SCHEMA["molexp"] == frozenset({"workspace"}) diff --git a/tests/test_stack.py b/tests/test_stack.py new file mode 100644 index 0000000..f65e0a7 --- /dev/null +++ b/tests/test_stack.py @@ -0,0 +1,1699 @@ +"""FastMCP composition: core + namespaced provider mounts.""" + +from __future__ import annotations + +import ast +import inspect +import json +import subprocess +import sys +from collections.abc import Mapping, Sequence +from dataclasses import dataclass, field +from pathlib import Path + +import pytest +from fastmcp import FastMCP +from mcp.types import ToolAnnotations + +from molmcp import CollectionIndex, create_plane, create_stack, runtime, server +from molmcp import harness as harness_module +from molmcp.components import ( + ALLOWED_REQUIRES, + BundleSpec, + ComponentKind, + ComponentSpec, + HarnessCatalog, +) +from molmcp.components.locator import LocatorError +from molmcp.config import AppConfig, ConfigurationError +from molmcp.provider_worker.worker import WorkerProvider +from molmcp.settings import HarnessSource, Settings + + +class _Vis: + name = "molvis" + + def register(self, mcp: FastMCP) -> None: + @mcp.tool(annotations=ToolAnnotations(read_only_hint=True)) + def open() -> str: + """Open a viewer session.""" + return "session" + + +async def test_stack_namespaces_provider_tools(): + stack = create_stack( + collection=CollectionIndex([]), + providers=[_Vis()], + discover_entry_points=False, + ) + assert stack.name == "molcrafts" + names = {tool.name for tool in await stack.list_tools()} + assert "packages" in names + assert "open" in names + assert "molvis_open" in names + + +async def test_stack_disable_skips_mount(): + stack = create_stack( + collection=CollectionIndex([]), + providers=[_Vis()], + disable=["molvis"], + discover_entry_points=False, + ) + names = {tool.name for tool in await stack.list_tools()} + assert "molvis_open" not in names + assert "packages" in names + + +async def test_single_provider_plane_stays_bare(): + server = create_plane( + "molvis", + provider=_Vis(), + discover_entry_points=False, + ) + assert server.name == "molvis" + names = {tool.name for tool in await server.list_tools()} + assert names == {"open"} + + +# --- autonomous harness wiring (spec 08) ---------------------------------- +# +# Every outbound seam create_stack could reach for is faked here: no git, no +# network, no environment. Each seam is patched on the module that *names* +# it, which is now two modules: ``molmcp.harness`` holds the checkout arms +# and every collaborator they construct (Activation, ImmutableGitStore, +# GitHubTransport, load_harness_catalog, WorkerProvider), while +# ``molmcp.server`` keeps what create_stack itself calls (load_settings, +# build_collection, discover_providers). Each arm resolves its collaborators +# from its own module globals, so a name patched on the module that merely +# imports that arm would never be read. + +_SHA = "0123456789abcdef0123456789abcdef01234567" +#: A second, distinct commit. One SHA per source is the whole point of a +#: per-source pointer file: two named sources may be activated at two +#: different commits, and a seam holding one ``current`` for all of them +#: could not express that at all. +_OTHER_SHA = "fedcba9876543210fedcba9876543210fedcba98" +_SOURCE = HarnessSource(name="official", locator="molcrafts/harness@main") +_OTHER = HarnessSource(name="private", locator="acme/tooling@trunk") +_CAPABILITIES = frozenset({"provider-sdk", "harness-catalog"}) +_SKILL = ComponentSpec( + kind=ComponentKind.SKILL, + name="daily", + id="skill.daily", + path="skills/daily.md", +) + +#: One recorded call: positional arguments, then keyword arguments. +_Call = tuple[tuple[object, ...], dict[str, object]] + + +def _config(tmp_path: Path) -> AppConfig: + """Config whose ``cache_dir`` is the already-resolved root of the store.""" + return AppConfig.from_dict( + {"schema_version": "2", "cache_dir": str(tmp_path / "cache")}, + workspace_root=tmp_path, + ) + + +def _catalog( + *components: ComponentSpec, sha: str = _SHA, component_root: str = "" +) -> HarnessCatalog: + """A real catalog: the two required bundles plus *components*. + + ``sha`` is the commit the catalog claims to describe. It matters only + when two sources are activated at two different commits, because the SHA + is the one argument a faked ``load_harness_catalog`` can tell two + checkouts apart by — every checkout in this suite shares one tree. + + ``component_root`` defaults to the absent key, so every call site written + before it describes a rootless catalog and reads exactly as it did. + """ + leaves = (_SKILL, *components) + ids = tuple(spec.id for spec in leaves) + return HarnessCatalog( + sha=sha, + requires=(), + components=leaves, + bundles=( + BundleSpec(name="daily", members=ids), + BundleSpec(name="dev", members=ids), + ), + component_root=component_root, + ) + + +def _provider_component( + path: str = "providers/demo/plane.py", + *, + name: str = "demo", + entrypoint: str = "demo.plane:DemoProvider", +) -> ComponentSpec: + """A checkout provider row whose ``id`` differs from its ``name``. + + ``entrypoint`` is the only field two sources' ``provider.demo`` rows can + differ in *and* have the difference reach an assertion: ``name`` and + ``id`` are the contested key itself, and ``path`` resolves through the + one shared fake tree, so two spellings of it land on one import root. + """ + return ComponentSpec( + kind=ComponentKind.PROVIDER, + name=name, + id=f"provider.{name}", + path=path, + entrypoint=entrypoint, + ) + + +def _overlay_component( + path: str = "overlays/demo.py", + *, + name: str = "demo", + entrypoint: str = "demo:make_overlay", +) -> ComponentSpec: + """A checkout overlay row — one seed for the overlay arm to hand on. + + Nothing here is ever imported: ``_wire`` fakes the loader itself, so the + row only has to be a real ``ComponentSpec`` of the kind the overlay fold + keeps. What a seed is imported *from* is this file's subject, and that + base is recorded rather than resolved; the real loader is driven against + a real tree in ``tests/test_runtime.py``. + """ + return ComponentSpec( + kind=ComponentKind.OVERLAY, + name=name, + id=f"overlay.{name}", + path=path, + entrypoint=entrypoint, + ) + + +def _checkout(tmp_path: Path, *, component_root: str = "") -> Path: + """A tree holding ``providers/demo/`` as a directory and a module in it. + + ``component_root`` plants that directory under the catalog-declared root + instead of at the top of the tree, so ``_import_root``'s + a-directory-is-used-as-it-stands branch answers about the base the fold + resolved rather than about the tree. The return value stays the *tree* — + what a store hands back — because that is what ``_wire`` is given. + """ + tree = tmp_path / "tree" + base = tree.joinpath(*component_root.split("/")) if component_root else tree + package = base / "providers" / "demo" + package.mkdir(parents=True) + (package / "plane.py").write_text("", encoding="utf-8") + return tree + + +#: ``user.name`` / ``user.email`` for the one commit ``_git_checkout`` makes. +#: Passed per invocation rather than configured, so no developer's global git +#: identity is read and none is written into ``tmp_path``. +_GIT_IDENTITY = ( + "-c", + "user.name=molmcp tests", + "-c", + "user.email=tests@molmcp.invalid", +) + + +def _git_checkout(root: Path) -> Path: + """Create *root* as a real one-commit git repository and return it. + + A ``path`` source is complete only when it names a checkout, so the + tests for the completed case need an actual repository rather than a + directory: ``git init`` is the whole difference between this helper and + :func:`_checkout` above, and it is the difference the locator now reads. + + Mirrored from ``tests/test_components/test_git.py``'s ``_init`` / + ``_commit`` rather than imported from it. Those are private names in a + module this change does not touch, and importing them would make a + refactor of the transport's own tests break the composition tests; the + three lines are cheaper than the coupling. + """ + root.mkdir(parents=True, exist_ok=True) + _run_git(root, "init", "-q", "--initial-branch=main") + (root / "harness.toml").write_text("", encoding="utf-8") + _run_git(root, "add", "-A") + _run_git(root, *_GIT_IDENTITY, "commit", "--no-gpg-sign", "-q", "-m", "first") + return root + + +def _run_git(root: Path, *args: str) -> None: + subprocess.run( + ["git", "-C", str(root), *args], + check=True, + capture_output=True, + text=True, + ) + + +class _Marker: + """An in-tree provider that registers one identifiable tool.""" + + def __init__(self, name: str, tool: str = "intree") -> None: + self.name = name + self._tool = tool + + def register(self, mcp: FastMCP) -> None: + @mcp.tool(name=self._tool, annotations=ToolAnnotations(read_only_hint=True)) + def marker() -> str: + """Prove which provider object was mounted under this namespace.""" + return self._tool + + +class _FakeWorker: + """Stand-in for ``WorkerProvider`` recording the constructor mapping.""" + + def __init__(self, *, name: str, entrypoint: str, path: str | Path) -> None: + self.name = name + self.entrypoint = entrypoint + self.path = path + + def probe(self) -> bool: + return True + + def register(self, mcp: FastMCP) -> None: + @mcp.tool(name="worker", annotations=ToolAnnotations(read_only_hint=True)) + def worker() -> str: + """Prove the checkout worker, not the in-tree plane, was mounted.""" + return self.name + + def __getattr__(self, item: str) -> object: + if item == "close": + raise AssertionError("spec 08 must not reach for WorkerProvider.close") + raise AttributeError(item) + + +class _FakeTransport: + """GitHub transport stand-in that refuses to touch the network.""" + + def resolve_commit(self, owner: str, repo: str, ref: str | None) -> str: + raise AssertionError("serve must not resolve a ref to a commit") + + def fetch_archive(self, owner: str, repo: str, sha: str) -> bytes: + raise AssertionError("serve must not fetch an archive") + + +class _FakeStore: + """Immutable store stand-in over a checkout that is already on disk.""" + + def __init__( + self, + root: str | Path, + transport: object, + *, + tree: Path | None, + published: bool, + ) -> None: + self.root = Path(root) + self.transport = transport + self._tree = tree + self._published = published + + def has(self, sha: str) -> bool: + return self._published + + def tree_path(self, sha: str) -> Path: + if not self._published or self._tree is None: + raise AssertionError(f"tree_path asked for unpublished {sha!r}") + return self._tree + + def publish(self, sha: str, *, owner: str, repo: str) -> Path: + raise AssertionError("serve must not clone a missing sha") + + +class _FakeActivation: + """Pointer view with a fixed ``current`` whose writes are all refused.""" + + def __init__(self, current: str | None) -> None: + self.current = current + self.staged: str | None = None + self.previous: str | None = None + + def stage(self, sha: str) -> None: + raise AssertionError(f"serve must not stage {sha!r}") + + def promote(self) -> None: + raise AssertionError("serve must not promote") + + def rollback(self) -> None: + raise AssertionError("serve must not roll back") + + +def _pointer_source(pointer: Path) -> str | None: + """Recover the source name a per-source pointer file belongs to. + + ``/harness.official.pointer`` is the source named ``official``. + The one shared ``/harness.pointer`` names no source and yields + ``None`` — unambiguously, because the store root beside it is a + *directory* named ``harness``, so no source name can produce that file. + """ + name = pointer.name + if not (name.startswith("harness.") and name.endswith(".pointer")): + return None + return name[len("harness.") : -len(".pointer")] or None + + +class _ActivationSeam: + """Stand-in for the ``Activation`` class; only ``bind`` is ever used. + + A scalar ``current`` answers the same SHA for every source, which is what + every single-source caller means and why they need no argument of their + own. A ``currents`` mapping answers per source, and the source is + recovered from the *pointer path* handed to :meth:`bind` rather than from + call order — so an arm that binds the right number of files in the wrong + order, or one that keeps binding a single shared pointer, cannot satisfy + it by accident. + """ + + def __init__( + self, + wiring: _Wiring, + current: str | None, + currents: Mapping[str, str | None] | None, + ) -> None: + self._wiring = wiring + self._current = current + self._currents = currents + + def bind( + self, + path: str | Path, + *, + store: object, + supported_capabilities: object, + ) -> _FakeActivation: + pointer = Path(path) + self._wiring.binds.append( + { + "path": pointer, + "store": store, + "supported_capabilities": supported_capabilities, + } + ) + return _FakeActivation(self._current_for(pointer)) + + def _current_for(self, pointer: Path) -> str | None: + """The SHA the source owning this pointer file is activated at.""" + if self._currents is None: + return self._current + source = _pointer_source(pointer) + if source is None: + raise AssertionError( + f"currents= names one SHA per source, but {pointer.name!r} " + "carries no source name: the arm is still binding one shared " + "pointer for every source" + ) + if source not in self._currents: + raise AssertionError( + f"currents= was never told about the source {source!r}; " + f"it names {sorted(self._currents)}" + ) + return self._currents[source] + + +class _RecordingCollection(CollectionIndex): + """Collection that counts its own lifecycle calls.""" + + def __init__(self) -> None: + super().__init__([]) + self.starts = 0 + self.closes = 0 + + def start(self) -> None: + self.starts += 1 + super().start() + + def close(self) -> None: + self.closes += 1 + super().close() + + +@dataclass +class _Wiring: + """What the faked seams saw during one ``create_stack`` call.""" + + settings: list[_Call] = field(default_factory=list) + transports: list[_Call] = field(default_factory=list) + stores: list[_FakeStore] = field(default_factory=list) + binds: list[dict[str, object]] = field(default_factory=list) + catalogs: list[dict[str, object]] = field(default_factory=list) + overlays: list[dict[str, object]] = field(default_factory=list) + workers: list[_FakeWorker] = field(default_factory=list) + built: list[dict[str, object]] = field(default_factory=list) + collections: list[_RecordingCollection] = field(default_factory=list) + discoveries: list[dict[str, object]] = field(default_factory=list) + + +def _wire( + monkeypatch: pytest.MonkeyPatch, + *, + harness: tuple[HarnessSource, ...] | None = None, + tree: Path | None = None, + current: str | None = None, + currents: Mapping[str, str | None] | None = None, + published: bool = True, + catalog: HarnessCatalog | None = None, + catalogs: Mapping[str, HarnessCatalog] | None = None, + entry_points: tuple[object, ...] = (), +) -> _Wiring: + """Fake every seam ``create_stack`` reaches out through and record it. + + ``current`` is one activated SHA for every named source; ``currents`` + names one per source, with ``None`` for a source that has nothing + activated. They are mutually exclusive: honouring both would mean the seam + picking one of two answers with nothing in the call saying which, so + passing both is refused here rather than resolved silently. + + ``catalog`` is likewise one catalog for every checkout, and ``catalogs`` + names one **per activated SHA** — keyed by commit rather than by source + name because the SHA is the only argument that reaches a catalog load + (``load_harness_catalog(tree, sha, capabilities)``) and every checkout in + this suite shares one faked tree. Two sources therefore need two distinct + ``currents`` before they can have two distinct catalogs, which is the + real relationship: what a source contributes follows from the commit it + is activated at. + + ``_session_capability_overlays`` is faked alongside the git seams rather + than left real, because it is one too: it puts a checkout directory on + ``sys.path`` and imports out of it, in this process, for the rest of the + run. Faking it records the *base* create_stack chose, which is this + file's share of the overlay arm — what a loader then does with a base + belongs to ``tests/test_runtime.py``, where the real function runs + against a real tree. + """ + if current is not None and currents is not None: + raise TypeError( + "_wire takes current= (one SHA for every source) or currents= " + "(one SHA per source name), never both" + ) + if catalog is not None and catalogs is not None: + raise TypeError( + "_wire takes catalog= (one catalog for every checkout) or " + "catalogs= (one catalog per activated SHA), never both" + ) + wiring = _Wiring() + resolved_catalog = catalog if catalog is not None else _catalog() + + def load_settings(*args: object, **kwargs: object) -> Settings: + wiring.settings.append((args, kwargs)) + return Settings(harness=tuple(harness or ())) + + def github_transport(*args: object, **kwargs: object) -> _FakeTransport: + wiring.transports.append((args, kwargs)) + return _FakeTransport() + + def immutable_git_store(root: str | Path, transport: object) -> _FakeStore: + made = _FakeStore(root, transport, tree=tree, published=published) + wiring.stores.append(made) + return made + + def load_harness_catalog( + tree: str | Path, + sha: str, + supported_capabilities: object, + ) -> HarnessCatalog: + wiring.catalogs.append( + {"tree": Path(tree), "sha": sha, "capabilities": supported_capabilities} + ) + if catalogs is None: + return resolved_catalog + if sha not in catalogs: + raise AssertionError( + f"catalogs= names one catalog per activated SHA and was " + f"never told about {sha!r}; it names {sorted(catalogs)}" + ) + return catalogs[sha] + + def session_capability_overlays( + seeds: Sequence[ComponentSpec], base: Path + ) -> tuple[object, ...]: + wiring.overlays.append({"seeds": tuple(seeds), "base": base}) + return () + + def worker_provider(*, name: str, entrypoint: str, path: str | Path) -> _FakeWorker: + made = _FakeWorker(name=name, entrypoint=entrypoint, path=path) + wiring.workers.append(made) + return made + + def build_collection( + config: object, + registry: object = None, + *, + extras: Sequence[object] = (), + ) -> _RecordingCollection: + wiring.built.append( + {"config": config, "registry": registry, "extras": tuple(extras)} + ) + collection = _RecordingCollection() + wiring.collections.append(collection) + return collection + + def discover_providers( + *, + failures: list[dict[str, str]] | None = None, + only_available: bool = False, + ) -> list[object]: + wiring.discoveries.append({"only_available": only_available}) + return list(entry_points) + + monkeypatch.setattr(server, "load_settings", load_settings) + monkeypatch.setattr(harness_module, "GitHubTransport", github_transport) + monkeypatch.setattr(harness_module, "ImmutableGitStore", immutable_git_store) + monkeypatch.setattr( + harness_module, "Activation", _ActivationSeam(wiring, current, currents) + ) + monkeypatch.setattr(harness_module, "load_harness_catalog", load_harness_catalog) + monkeypatch.setattr(harness_module, "WorkerProvider", worker_provider) + monkeypatch.setattr( + server, "_session_capability_overlays", session_capability_overlays + ) + monkeypatch.setattr(server, "build_collection", build_collection) + monkeypatch.setattr(server, "discover_providers", discover_providers) + return wiring + + +async def _tool_names(stack: FastMCP) -> set[str]: + return {tool.name for tool in await stack.list_tools()} + + +async def _tool_name_list(stack: FastMCP) -> list[str]: + """Every composed tool name *with* its multiplicity. + + :func:`_tool_names` collapses a name mounted twice into one entry, so it + cannot tell "one plane named ``demo``" from "two planes mounted under one + ``demo`` namespace". Counting needs the list. + """ + return [tool.name for tool in await stack.list_tools()] + + +# -- arm gating ------------------------------------------------------------- + + +def test_dual_injection_never_consults_the_harness_locator(tmp_path, monkeypatch): + """Both arms injected: the locator is not read, bound, or catalogued.""" + wiring = _wire(monkeypatch, harness=(_SOURCE,)) + create_stack( + collection=CollectionIndex([]), + providers=[_Vis()], + config=_config(tmp_path), + ) + assert wiring.settings == [] + assert wiring.binds == [] + assert wiring.catalogs == [] + + +async def test_an_empty_source_list_serves_exactly_like_today(tmp_path, monkeypatch): + """No source named: no bind, no extras, entry points then ``disable=``. + + The empty *list* is the un-harnessed install — the one configuration + that must keep serving exactly as it did before a harness existed. + """ + wiring = _wire( + monkeypatch, + harness=(), + entry_points=(_Marker("demo"), _Marker("other")), + ) + stack = create_stack(config=_config(tmp_path), disable=["other"]) + assert wiring.binds == [] + assert wiring.catalogs == [] + assert wiring.built[0]["extras"] == () + assert wiring.discoveries == [{"only_available": True}] + names = await _tool_names(stack) + assert "demo_intree" in names + assert "other_intree" not in names + + +def test_an_empty_locator_cannot_construct(): + """Half-authored coordinate entries are no longer representable.""" + with pytest.raises((LocatorError, ValueError)): + HarnessSource(name="mine", locator="") + + +def test_a_relative_locator_cannot_construct(): + with pytest.raises((LocatorError, ValueError)): + HarnessSource(name="mine", locator="./checkout") + + +def test_a_name_only_entry_cannot_construct(): + with pytest.raises(TypeError): + HarnessSource(name="mine") + + +def test_a_github_locator_without_a_ref_is_servable(monkeypatch): + """``owner/repo`` with no ``@ref`` is a complete GitHub origin.""" + source = HarnessSource(name="official", locator="molcrafts/harness") + assert source.ref == "" + _wire(monkeypatch, harness=(source,)) + assert server._harness_locator() == (source,) + + +def test_two_complete_sources_are_returned_in_file_order(monkeypatch): + """Order is the file's order — the contract later resolution inherits.""" + _wire(monkeypatch, harness=(_SOURCE, _OTHER)) + assert server._harness_locator() == (_SOURCE, _OTHER) + + +def test_two_sources_still_bind_exactly_one_store_root(tmp_path, monkeypatch): + """Several named sources, one store: no per-source root is introduced. + + ``ImmutableGitStore`` already records provenance per SHA and refuses a + SHA claimed by a second repository, so a second root would buy nothing + and would strand every already-published tree. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + assert len(wiring.stores) == 1 + assert wiring.stores[0].root == config.cache_dir / "harness" + + +def test_two_sources_bind_one_activation_pointer_each(tmp_path, monkeypatch): + """One store above, one *pointer file per source* here — the other half. + + This is the half of the old single-store test that per-source activation + breaks, split out rather than deleted so the store's reason keeps its own + test. A pointer is not shareable the way a store is: the record + ``Activation`` binds holds one ``active`` SHA, so a second source folded + into ``/harness.pointer`` would either overwrite the first's + commit or be overwritten by it. The file name carries the source instead, + and the store keeps its one root because it is keyed by SHA and needs no + such name. + + The paths are asserted as an ordered list, so a bind that lands the right + number of files under the wrong names fails here rather than passing on a + count. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.official.pointer", + config.cache_dir / "harness.private.pointer", + ] + + +# -- completeness is the locator, not a coordinate triple ------------------- +# +# A GitHub locator is complete without a path. A local locator is an +# absolute or ``~/`` path that names a checkout. Relative spellings cannot +# construct. + + +def test_a_local_source_with_a_locator_path_is_complete(tmp_path, monkeypatch): + """A checkout on disk is an origin; GitHub coordinates stay derived empty.""" + source = HarnessSource( + name="mine", locator=str(_git_checkout(tmp_path / "checkout")) + ) + _wire(monkeypatch, harness=(source,)) + + assert server._harness_locator() == (source,) + + +async def test_a_local_source_reaches_the_activation_arm_and_serves( + tmp_path, monkeypatch +): + """The whole composition, not just the locator: a local entry serves. + + ``_harness_locator`` returning the source is necessary and not + sufficient — the entry has to travel the same arm a remote one does. The + pointer bind is the evidence it did, and the core tool is the evidence + the stack came up rather than raising on the way. + """ + config = _config(tmp_path) + source = HarnessSource( + name="mine", locator=str(_git_checkout(tmp_path / "checkout")) + ) + wiring = _wire(monkeypatch, harness=(source,)) + + stack = create_stack(config=config) + + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.mine.pointer" + ] + assert "packages" in await _tool_names(stack) + + +@pytest.mark.parametrize("directory", ["gone", "plain"]) +def test_a_local_path_that_is_not_a_checkout_is_refused_by_the_locator( + tmp_path, monkeypatch, directory: str +): + """A path naming no checkout is the local half-authored coordinate. + + Two ways to get one, and they are one failure: the directory is not + there at all (``gone``), or it is there and is not a repository + (``plain``) — an operator who typed the parent, or the checkout before + cloning into it. Both are refused *here*, beside the remote entry's + missing ``ref``, rather than later as a ``GitError`` out of a transport: + the settings file is what is wrong, and the message has to say which + entry and which path so there is somewhere to go and fix it. + """ + root = tmp_path / directory + if directory == "plain": + root.mkdir() + source = HarnessSource(name="mine", locator=str(root)) + _wire(monkeypatch, harness=(source,)) + + with pytest.raises(ConfigurationError) as excinfo: + server._harness_locator() + + message = str(excinfo.value) + assert "mine" in message + assert str(root) in message + + +def test_an_unusable_local_path_refuses_before_anything_is_bound(tmp_path, monkeypatch): + """Refused whole: no store, no pointer, no transport for the bad entry. + + The complement of the parametrized test above. It proves the raise comes + out of the locator; this one proves nothing downstream of the locator ran + first, which is what "fails at the same place, not later inside the + transport" costs if it is not true — a half-bound cache directory for a + settings file that was never servable. + """ + source = HarnessSource(name="mine", locator=str(tmp_path / "gone")) + wiring = _wire(monkeypatch, harness=(source,)) + + with pytest.raises(ConfigurationError): + create_stack(config=_config(tmp_path)) + + assert wiring.stores == [] + assert wiring.binds == [] + assert wiring.catalogs == [] + + +def test_a_local_and_a_remote_source_are_complete_side_by_side(tmp_path, monkeypatch): + """One list, two origins, file order kept — the mixed install. + + Origin is read per entry. A rule that picked one shape for the whole + list would either reject the local entry or stop checking the remote + one's coordinates. + """ + local = HarnessSource( + name="mine", locator=str(_git_checkout(tmp_path / "checkout")) + ) + _wire(monkeypatch, harness=(_SOURCE, local)) + + assert server._harness_locator() == (_SOURCE, local) + + +# -- the reader itself, over a real settings file --------------------------- +# +# Every test above fakes ``load_settings`` through the ``_wire`` seam, so a +# reader that cannot read the type it is handed passes all of them. These two +# call the real ``_harness_locator`` against a settings file on disk, under a +# temporary home so no developer's own ``~/.molmcp`` can reach the assertion. + + +def _fake_home(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Point ``~`` at a temporary tree and return it. + + Both spellings of "the user's home" are aimed at the same directory: + :meth:`Path.home`, which is how this package finds it, and the ``HOME`` / + ``USERPROFILE`` environment :func:`os.path.expanduser` consults — + ``Path.expanduser`` delegates to that function and does **not** go through + ``Path.home``. Pinning both keeps the ``~`` tests below on the behaviour + (a home-relative path names one directory in every session) rather than on + which of the two APIs an expansion is written with. + """ + home = tmp_path / "home" + home.mkdir(parents=True, exist_ok=True) + monkeypatch.setattr(Path, "home", staticmethod(lambda: home)) + monkeypatch.setenv("HOME", str(home)) + monkeypatch.setenv("USERPROFILE", str(home)) + return home + + +def _working_directory(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path: + """Stand the process in a temporary project directory and return it. + + This is the directory an MCP client's ``molmcp serve`` inherits — one of + many, differing per session, and the thing a ``path`` entry in the shared + settings file may not be read against. + """ + project = tmp_path / "project" + project.mkdir(parents=True, exist_ok=True) + monkeypatch.chdir(project) + return project + + +def _home_settings( + tmp_path: Path, monkeypatch: pytest.MonkeyPatch, data: dict[str, object] +) -> Path: + """Point ``~`` and the working directory at hermetic temporary trees. + + Returns the settings file it wrote, so a caller can read the bytes back + and check that serving left them alone. + """ + home = _fake_home(tmp_path, monkeypatch) + (home / ".molmcp").mkdir(parents=True) + settings_file = home / ".molmcp" / "settings.json" + settings_file.write_text(json.dumps(data), encoding="utf-8") + _working_directory(tmp_path, monkeypatch) + return settings_file + + +def test_the_real_locator_reads_the_named_sources_off_disk(tmp_path, monkeypatch): + """The unfaked reader over a real file, in file order.""" + _home_settings( + tmp_path, + monkeypatch, + { + "harness": [ + {"name": "official", "locator": "molcrafts/harness@main"}, + {"name": "private", "locator": "acme/tooling@trunk"}, + ] + }, + ) + assert server._harness_locator() == (_SOURCE, _OTHER) + + +def test_the_real_locator_reads_an_empty_settings_file_as_no_harness( + tmp_path, monkeypatch +): + """The unfaked reader on a stock install: ``()``, not an error.""" + _home_settings(tmp_path, monkeypatch, {}) + assert server._harness_locator() == () + + +def test_the_real_locator_accepts_a_path_entry_off_disk(tmp_path, monkeypatch): + """The local origin, end to end: settings file to servable source. + + Every faked-seam test above hands ``_harness_locator`` a + ``HarnessSource`` the test itself constructed, so a reader that cannot + round-trip the ``path`` key through JSON passes all of them — the exact + failure this section exists for. Here the entry is a dict in a file and + the checkout is a real repository. + """ + checkout = _git_checkout(tmp_path / "checkout") + _home_settings( + tmp_path, + monkeypatch, + {"harness": [{"name": "mine", "locator": str(checkout)}]}, + ) + + assert server._harness_locator() == ( + HarnessSource(name="mine", locator=str(checkout)), + ) + + +def test_harness_coordinates_are_gone_from_server(): + assert not hasattr(server, "_HARNESS_KEYS") + assert not hasattr(harness_module, "HARNESS_COORDINATES") + + +# -- local locators: absolute, ``~/``, never relative ----------------------- +# +# Relative paths cannot construct a ``HarnessSource``. ``~/harness`` and an +# absolute checkout remain servable; the stored locator string is not +# rewritten on the way through the reader. + + +async def test_a_home_relative_source_reaches_the_activation_arm_and_serves( + tmp_path, monkeypatch +): + """``~/harness`` travels the same arm an absolute path does. + + Being accepted by the locator is necessary and not sufficient — the + expansion has to hold all the way through activation. The pointer bind is + the evidence the entry got there, and the core tool is the evidence the + stack came up rather than raising on the way. + """ + config = _config(tmp_path) + home = _fake_home(tmp_path, monkeypatch) + _working_directory(tmp_path, monkeypatch) + _git_checkout(home / "harness") + wiring = _wire( + monkeypatch, harness=(HarnessSource(name="mine", locator="~/harness"),) + ) + + stack = create_stack(config=config) + + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.mine.pointer" + ] + assert "packages" in await _tool_names(stack) + + +def test_a_home_relative_path_resolves_under_home_not_the_working_directory( + tmp_path, monkeypatch +): + """The expansion is home's, and it is not a search path. + + The only checkout on disk sits at ``harness`` under the *working + directory* and home is empty, so the entry names nothing and is refused — + proof that the accepted case above was the home expansion rather than a + relative read that happened to find a repository. + """ + _fake_home(tmp_path, monkeypatch) + project = _working_directory(tmp_path, monkeypatch) + _git_checkout(project / "harness") + _wire(monkeypatch, harness=(HarnessSource(name="mine", locator="~/harness"),)) + + with pytest.raises(ConfigurationError) as excinfo: + server._harness_locator() + + assert "mine" in str(excinfo.value) + + +def test_the_real_locator_serves_a_home_relative_path_without_rewriting_it( + tmp_path, monkeypatch +): + """The unfaked reader over a real file: accepted, and the file untouched. + + ``_harness_locator`` is the one step of a serve that opens + ``~/.molmcp/settings.json``, so it is the one step that could normalise + the entry on the way past. It must not: the stored string is the + operator's, an expansion belongs to the session doing the resolving, and + a machine's absolute path written back into a file that syncs between + machines is a different bug in the same family. + """ + settings_file = _home_settings( + tmp_path, monkeypatch, {"harness": [{"name": "mine", "locator": "~/harness"}]} + ) + before = settings_file.read_bytes() + # ``/.molmcp/settings.json`` — read back off the helper's own answer + # rather than respelled here, so the checkout lands under whatever ``~`` + # was pointed at. + _git_checkout(settings_file.parent.parent / "harness") + + assert server._harness_locator() == ( + HarnessSource(name="mine", locator="~/harness"), + ) + + assert settings_file.read_bytes() == before + assert "~/harness" in settings_file.read_text(encoding="utf-8") + + +def test_the_real_locator_serves_an_absolute_path_without_rewriting_it( + tmp_path, monkeypatch +): + """The spelling that was always accepted, still accepted and still verbatim. + + The control for the test above: whatever the new rule does to a moving + path, an absolute entry keeps serving and its string keeps its bytes. + """ + checkout = _git_checkout(tmp_path / "checkout") + settings_file = _home_settings( + tmp_path, monkeypatch, {"harness": [{"name": "mine", "locator": str(checkout)}]} + ) + before = settings_file.read_bytes() + + assert server._harness_locator() == ( + HarnessSource(name="mine", locator=str(checkout)), + ) + + assert settings_file.read_bytes() == before + stored = json.loads(settings_file.read_text(encoding="utf-8")) + assert stored["harness"][0]["locator"] == str(checkout) + + +@pytest.mark.parametrize( + ("sources", "currents", "pointers", "shas"), + [ + pytest.param( + (_SOURCE,), + {_SOURCE.name: None}, + ("harness.official.pointer",), + (), + id="the-one-source-has-nothing-activated", + ), + pytest.param( + (_SOURCE, _OTHER), + {_SOURCE.name: None, _OTHER.name: _OTHER_SHA}, + ("harness.official.pointer", "harness.private.pointer"), + (_OTHER_SHA, _OTHER_SHA), + id="the-first-of-two-sources-has-nothing-activated", + ), + ], +) +async def test_absent_current_falls_back_without_resolving_or_promoting( + tmp_path, + monkeypatch, + sources: tuple[HarnessSource, ...], + currents: Mapping[str, str | None], + pointers: tuple[str, ...], + shas: tuple[str, ...], +): + """A source with no current SHA serves the unset fallback — even mixed. + + Binding is a *read*: a source with nothing activated is an empty record, + not a skipped file, so every named source is bound whatever its neighbour + is at. Only the ones that came back with a SHA go on to a catalog, and + the second case is the mixed one — the unactivated source comes *first*, + so an arm that stopped at the first empty record would serve the second + source's commit as nothing at all. + + The fallback assertions hold in both cases because the activated source's + catalog declares no overlay and no provider: extras stay empty, entry + points are still enumerated, and the in-tree ``demo`` plane still mounts. + + This does not claim which source got which SHA. With one SHA in play the + mapping is unfalsifiable from here — the two names could be swapped and + the same one catalog read would follow. It is pinned where the pointer + files are real, in ``tests/test_harness.py::TestActivatedCheckouts``, and + the two-commit case below is the composition-side half of it. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=sources, + currents=currents, + tree=_checkout(tmp_path), + entry_points=(_Marker("demo"),), + ) + stack = create_stack(config=config) + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / name for name in pointers + ] + assert [entry["sha"] for entry in wiring.catalogs] == list(shas) + assert wiring.built[0]["extras"] == () + assert wiring.discoveries == [{"only_available": True}] + assert "demo_intree" in await _tool_names(stack) + + +def test_two_activated_sources_are_each_read_at_their_own_commit(tmp_path, monkeypatch): + """Two sources, two commits: each checkout is read at its own pointer's SHA. + + Both sources are activated, at two *distinct* commits, so the pairing is + falsifiable here in a way it is not with one SHA in play: the seam answers + by pointer file name, and the catalog reads are asserted as an ordered + list, one per arm per checkout. An arm that read ``harness.private``'s + answer for ``official`` flips both halves of that list, and an arm that + still binds one shared pointer never gets an answer at all — neither + failure is visible to a count or to a set. + + What is claimed is exactly ``pointer file -> SHA -> the catalog read for + that checkout``. The ``source`` *label* the checkout carries is not + claimed: every checkout in this suite shares one faked tree, so a + correctly ordered pair of checkouts wearing each other's names would read + the same catalogs in the same order. That label is pinned in + ``tests/test_harness.py::TestActivatedCheckouts``, where the pointer files + are real. + + Two reads per commit is not redundancy to be optimised away: the overlay + arm and the provider arm each read the catalog for themselves, so an arm + that does not run never pays for a catalog it would not use. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + currents={_SOURCE.name: _SHA, _OTHER.name: _OTHER_SHA}, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.official.pointer", + config.cache_dir / "harness.private.pointer", + ] + assert [entry["sha"] for entry in wiring.catalogs] == [ + _SHA, + _OTHER_SHA, + _SHA, + _OTHER_SHA, + ] + + +def test_one_activated_source_and_one_without_are_answered_apart(tmp_path, monkeypatch): + """Two named sources, one activated: two binds, one source's catalogs. + + This is the first test that needs the activation pointer to be per + source, and the first that a seam holding a single ``current`` could not + express: ``official`` is activated at :data:`_SHA` while ``private`` has + nothing activated at all. Both pointers are still bound — binding is a + read, and an unactivated source is an empty record rather than a skipped + one — but only ``official`` has a tree to read a catalog from, so the two + reads the two arms make are both for its SHA. + + Three ways of getting this wrong die here: an arm still binding one + shared ``harness.pointer`` (the seam cannot recover a source name from + that file and says so), a seam ignoring ``currents`` for the scalar + ``current`` (no catalog read at all), and a seam answering one SHA for + every source (two checkouts, so four reads rather than two). + + One axis is deliberately *not* claimed: with a single source activated, + swapping which name holds ``_SHA`` still yields one checkout at ``_SHA``, + so nothing here pins name to SHA — the mapping is written in the opposite + order to the source list to discourage a positional reading, not to prove + one impossible. That pairing is pinned where the pointer files are real, + in ``tests/test_harness.py::TestActivatedCheckouts``, which is also the + only place the real ``Activation.bind`` and ``load_harness_catalog`` are + exercised at all: ``_wire`` fakes both here + (``notes.md:faked-seam-hides-broken-reader``), so this test is evidence + about what ``create_stack`` asks for and none about what answers it. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + currents={_OTHER.name: None, _SOURCE.name: _SHA}, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.official.pointer", + config.cache_dir / "harness.private.pointer", + ] + assert [entry["sha"] for entry in wiring.catalogs] == [_SHA, _SHA] + + +def test_current_missing_from_the_store_names_that_sha(tmp_path, monkeypatch): + """An activated SHA with no tree is an error, never a silent re-clone.""" + _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + published=False, + ) + with pytest.raises(ConfigurationError) as excinfo: + create_stack(config=_config(tmp_path)) + assert _SHA in str(excinfo.value) + + +async def test_injected_collection_still_runs_the_provider_git_arm( + tmp_path, monkeypatch +): + """Skip is per owner: an injected collection only skips the overlay arm. + + One arm runs, so the reads are one per activated source rather than two + — the count that would drop back to one is an arm that folded a single + checkout out of two named sources. + """ + tree = _checkout(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=tree, + catalog=_catalog(_provider_component()), + ) + stack = create_stack(collection=CollectionIndex([]), config=_config(tmp_path)) + assert "demo_worker" in await _tool_names(stack) + assert wiring.built == [] + assert len(wiring.catalogs) == 2 + + +async def test_injected_providers_still_run_the_overlay_git_arm(tmp_path, monkeypatch): + """Injected providers skip only the provider arm and still pass disable=. + + The surviving overlay arm reads one catalog per activated source, the + mirror of the provider-arm case above. + """ + tree = _checkout(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=tree, + catalog=_catalog(_provider_component()), + ) + stack = create_stack( + providers=[_Marker("demo")], + config=_config(tmp_path), + disable=["demo"], + ) + assert "demo_intree" not in await _tool_names(stack) + assert wiring.workers == [] + assert wiring.discoveries == [] + assert len(wiring.catalogs) == 2 + assert len(wiring.built) == 1 + + +async def test_entry_point_discovery_off_is_not_a_provider_git_arm( + tmp_path, monkeypatch +): + """``discover_entry_points=False`` with no providers mounts nothing.""" + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + catalog=_catalog(_provider_component()), + ) + stack = create_stack( + collection=CollectionIndex([]), + config=_config(tmp_path), + discover_entry_points=False, + ) + assert [name for name in await _tool_names(stack) if name.startswith("demo_")] == [] + assert wiring.settings == [] + assert wiring.binds == [] + assert wiring.workers == [] + + +# -- named bind ------------------------------------------------------------- + + +def test_named_store_and_pointer_hang_off_the_resolved_cache_root( + tmp_path, monkeypatch +): + """One store at ``/harness``, a pointer per source beside it. + + The transport is constructed once for any number of sources — + ``GitHubTransport`` takes ``(owner, repo)`` per call, so a second source + in a second repository needs no second instance — and its constructor is + still called with nothing, the token staying where it already lives. + + Every bind is handed that same one store *object*, not merely an equal + root: the identity is what says the N pointers share one SHA-keyed store + rather than N stores that happen to agree on a path. + """ + config = _config(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + assert wiring.transports == [((), {})] + assert len(wiring.stores) == 1 + store = wiring.stores[0] + assert store.root == config.cache_dir / "harness" + assert isinstance(store.transport, _FakeTransport) + assert [bind["path"] for bind in wiring.binds] == [ + config.cache_dir / "harness.official.pointer", + config.cache_dir / "harness.private.pointer", + ] + assert [bind["store"] is store for bind in wiring.binds] == [True, True] + + +def test_unset_cache_dir_still_binds_under_the_resolved_default_root( + tmp_path, monkeypatch +): + """No ``cacheDir`` set is the common case, not a broken harness. + + ``AppConfig.cache_dir`` is ``None`` until somebody configures ``cacheDir``, + so reading it raw turns "I set the three harness keys" into an error for + the majority of users. The fallback to the default cache root already has + one home in :mod:`molmcp.runtime`; every per-source bind hangs off that + resolved root, and the one store beside them does too. + """ + config = AppConfig.from_dict({"schema_version": "2"}, workspace_root=tmp_path) + assert config.cache_dir is None + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=_checkout(tmp_path), + ) + create_stack(config=config) + resolved = runtime.resolved_cache_dir(config) + assert len(wiring.stores) == 1 + store = wiring.stores[0] + assert store.root == resolved / "harness" + assert [bind["path"] for bind in wiring.binds] == [ + resolved / "harness.official.pointer", + resolved / "harness.private.pointer", + ] + assert [bind["store"] is store for bind in wiring.binds] == [True, True] + + +def test_server_module_imports_nothing_from_discovery(): + """The cache root is AppConfig's; server.py stays out of discovery.""" + tree = ast.parse(Path(server.__file__).read_text(encoding="utf-8")) + modules: list[str] = [] + names: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + modules += [alias.name for alias in node.names] + names |= {alias.name for alias in node.names} + elif isinstance(node, ast.ImportFrom): + modules.append(node.module or "") + names |= {alias.name for alias in node.names} + assert [name for name in modules if "discovery" in name] == [] + assert "DiscoveryConfig" not in names + assert "default_cache_dir" not in names + + +def test_harness_module_imports_nothing_from_discovery(): + """The resolver inherits the shield, and the guard follows the code. + + ``server.py``'s own scan cannot see this: the harness arms moved to + ``molmcp.harness`` in ``3c407a8``, so a ``discovery`` import added there + would leave ``server.py`` clean and still breach the boundary. Both modules + reach the cache root through ``runtime.resolved_cache_dir``, which is the + one owner of that fallback precisely so neither has to import discovery. + """ + tree = ast.parse(Path(harness_module.__file__).read_text(encoding="utf-8")) + modules: list[str] = [] + names: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + modules += [alias.name for alias in node.names] + names |= {alias.name for alias in node.names} + elif isinstance(node, ast.ImportFrom): + modules.append(node.module or "") + names |= {alias.name for alias in node.names} + assert [name for name in modules if "discovery" in name] == [] + assert "DiscoveryConfig" not in names + assert "default_cache_dir" not in names + + +def test_the_locator_is_read_once_with_the_project_root(tmp_path, monkeypatch): + """``load_settings(Path.cwd())``: bare hides a project's harness keys.""" + config = _config(tmp_path) + tree = _checkout(tmp_path) + monkeypatch.chdir(tmp_path) + wiring = _wire(monkeypatch, harness=(_SOURCE,), current=_SHA, tree=tree) + create_stack(config=config) + assert len(wiring.settings) == 1 + args, kwargs = wiring.settings[0] + assert len(args) + len(kwargs) == 1 + assert (args[0] if args else kwargs["project_root"]) == Path.cwd() + + +# -- capabilities ----------------------------------------------------------- + + +def test_one_capability_object_reaches_bind_and_both_catalog_calls( + tmp_path, monkeypatch +): + """One frozenset object: every bind plus one catalog call per arm per source. + + Two arms over two sources is 2 x N calls, and the object handed to each + one is asserted by identity rather than equality. An equal-but-distinct + frozenset per source would pass an ``==`` check and would mean the + capability set had been rebuilt somewhere down the loop, which is the + thing this test exists to refuse. + """ + tree = _checkout(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + current=_SHA, + tree=tree, + catalog=_catalog(_provider_component()), + ) + create_stack(config=_config(tmp_path)) + assert harness_module.SUPPORTED_CAPABILITIES == _CAPABILITIES + capabilities = harness_module.SUPPORTED_CAPABILITIES + assert len(wiring.binds) == 2 + for bind in wiring.binds: + assert bind["supported_capabilities"] is capabilities + assert len(wiring.catalogs) == 4 + for call in wiring.catalogs: + assert call["capabilities"] is harness_module.SUPPORTED_CAPABILITIES + assert call["tree"] == tree + assert call["sha"] == _SHA + + +def test_supported_capabilities_is_a_subset_of_allowed_requires_not_an_alias(): + """Catalog grammar and runtime ability are two sets that happen to match.""" + assert harness_module.SUPPORTED_CAPABILITIES <= ALLOWED_REQUIRES + assert harness_module.SUPPORTED_CAPABILITIES is not ALLOWED_REQUIRES + + +# -- worker provider mapping and XOR --------------------------------------- + + +def test_worker_provider_is_named_by_component_name_not_id(tmp_path, monkeypatch): + """``name=`` is the EP name ``demo``; ``provider.demo`` is not a plane id.""" + spec = _provider_component() + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + catalog=_catalog(spec), + ) + create_stack(collection=CollectionIndex([]), config=_config(tmp_path)) + assert spec.id == "provider.demo" + assert [worker.name for worker in wiring.workers] == ["demo"] + assert wiring.workers[0].name != spec.id + + +def test_worker_provider_entrypoint_stays_an_unimported_string(tmp_path, monkeypatch): + """The entrypoint is carried as a string; the checkout is not imported.""" + spec = _provider_component() + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + catalog=_catalog(spec), + ) + create_stack(collection=CollectionIndex([]), config=_config(tmp_path)) + assert wiring.workers[0].entrypoint == spec.entrypoint + assert "demo.plane" not in sys.modules + assert "demo" not in sys.modules + + +@pytest.mark.parametrize("path", ["providers/demo/plane.py", "providers/demo"]) +def test_worker_provider_path_is_the_import_root_directory( + tmp_path, monkeypatch, path: str +): + """A file row gives its parent; a directory row gives that directory.""" + tree = _checkout(tmp_path) + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=tree, + catalog=_catalog(_provider_component(path=path)), + ) + create_stack(collection=CollectionIndex([]), config=_config(tmp_path)) + assert Path(wiring.workers[0].path) == tree / "providers" / "demo" + + +@pytest.mark.parametrize("path", ["providers/demo/plane.py", "providers/demo"]) +def test_rooted_worker_provider_path_is_under_the_component_root( + tmp_path, monkeypatch, path: str +): + """The sibling above, with ``component_root = "plugins/mol"`` declared. + + Both path shapes land on one directory again, and it is the one under + the root. The tree really holds ``plugins/mol/providers/demo``, so the + directory row takes ``_import_root``'s is-a-directory branch off the + folded base rather than falling back to a parent that happens to look + plausible. + """ + tree = _checkout(tmp_path, component_root="plugins/mol") + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=tree, + catalog=_catalog(_provider_component(path=path), component_root="plugins/mol"), + ) + create_stack(collection=CollectionIndex([]), config=_config(tmp_path)) + assert ( + Path(wiring.workers[0].path) == tree / "plugins" / "mol" / "providers" / "demo" + ) + + +async def test_checkout_wins_the_name_and_entry_point_only_planes_pass_through( + tmp_path, monkeypatch +): + """XOR against ``discover_providers(only_available=True)``, by EP name.""" + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + catalog=_catalog(_provider_component()), + entry_points=(_Marker("demo"), _Marker("other")), + ) + stack = create_stack(config=_config(tmp_path)) + names = await _tool_names(stack) + assert "demo_worker" in names + assert "demo_intree" not in names + assert "other_intree" in names + assert wiring.discoveries == [{"only_available": True}] + + +async def test_two_sources_declaring_one_plane_mount_it_once(tmp_path, monkeypatch): + """``provider.demo`` in two catalogs is one plane, and it is the first. + + This is the collision the fold exists for. ``ComponentSpec`` pins + ``id == f"provider.{name}"``, so two sources declaring ``provider.demo`` + are two planes named ``demo`` — and a plane name is the namespace its + tools mount under, so building both would mount twice under one + namespace and leave which one answers ``demo_worker`` to mount order. + + Both catalogs are still *read*: the fold collapses the id, it does not + skip a source. The two sources are activated at two different commits so + that they can declare two different rows at all, and the winner is + identified by ``entrypoint`` — the one field of a contested + ``provider.demo`` that can differ, since the name and the id are the + contested key itself. "First" is read off the chain + ``harness.official.pointer -> _SHA -> that catalog``, not off the + ``source`` label the checkout carries, which nothing in this file can see. + + The mount count is asserted on the tool-name *list*, because mounting + twice under one namespace may well leave a single name visible; the + number of workers constructed is the assertion that cannot be satisfied + by a shadowed second mount. + """ + first = _provider_component(entrypoint="demo.plane:OfficialProvider") + second = _provider_component(entrypoint="demo.plane:PrivateProvider") + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + currents={_SOURCE.name: _SHA, _OTHER.name: _OTHER_SHA}, + tree=_checkout(tmp_path), + catalogs={ + _SHA: _catalog(first, sha=_SHA), + _OTHER_SHA: _catalog(second, sha=_OTHER_SHA), + }, + ) + stack = create_stack(config=_config(tmp_path)) + assert len(wiring.catalogs) == 4 + assert [worker.name for worker in wiring.workers] == ["demo"] + assert wiring.workers[0].entrypoint == first.entrypoint + assert wiring.workers[0].entrypoint != second.entrypoint + assert (await _tool_name_list(stack)).count("demo_worker") == 1 + + +async def test_the_folded_name_set_excludes_a_plane_the_second_source_named( + tmp_path, monkeypatch +): + """The XOR is against every source's planes, not against the first's. + + ``test_checkout_wins_the_name_and_entry_point_only_planes_pass_through`` + covers this with one source. Here the checkout's ``demo`` comes from the + **second** source and the first declares something else entirely, so an + arm that built the entry-point exclusion set from the first catalog — or + from one checkout out of two — would let the in-tree ``demo`` plane + through and mount a second plane under that namespace. + + The exclusion set is the fold's, so both sources' kept planes are in it: + ``alpha`` and ``demo`` both mount, and only the unclaimed ``other`` + survives from the entry points. + """ + alpha = _provider_component(path="providers/alpha/plane.py", name="alpha") + demo = _provider_component() + wiring = _wire( + monkeypatch, + harness=(_SOURCE, _OTHER), + currents={_SOURCE.name: _SHA, _OTHER.name: _OTHER_SHA}, + tree=_checkout(tmp_path), + catalogs={ + _SHA: _catalog(alpha, sha=_SHA), + _OTHER_SHA: _catalog(demo, sha=_OTHER_SHA), + }, + entry_points=(_Marker("demo"), _Marker("other")), + ) + stack = create_stack(config=_config(tmp_path)) + assert [worker.name for worker in wiring.workers] == ["alpha", "demo"] + names = await _tool_names(stack) + assert "alpha_worker" in names + assert "demo_worker" in names + assert "demo_intree" not in names + assert "other_intree" in names + + +# -- overlay seed base ------------------------------------------------------ + + +@pytest.mark.parametrize( + ("component_root", "segments"), + [("", ()), ("plugins/mol", ("plugins", "mol"))], +) +def test_overlay_seeds_are_handed_the_folded_base_not_the_checkout_tree( + tmp_path, monkeypatch, component_root: str, segments: tuple[str, ...] +): + """``create_stack`` hands the overlay loader ``root_for``'s answer. + + This is the overlay half of the failure ``component_root`` exists to + make unreachable: the key applied in the provider arm and forgotten in + this one is half a harness, and half a harness is harder to diagnose + than one that resolves nothing, because the install looks like it works. + The provider half is asserted two sections above, and again over a real + tree in ``tests/test_harness.py``. + + The *recorded argument* is the assertion because the subject is + ``create_stack``'s composition — which directory it chose. What the + loader does with a base is the loader's contract, pinned in + ``tests/test_runtime.py`` against a real tree through the real function. + + The rootless case is an equality against the tree object itself, not a + prefix check, so a base carrying a ``.`` component or a trailing + separator fails it: today's rootless installs must resolve byte-identical + paths. + """ + tree = _checkout(tmp_path, component_root=component_root) + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=tree, + catalog=_catalog(_overlay_component(), component_root=component_root), + ) + create_stack(config=_config(tmp_path)) + assert [call["base"] for call in wiring.overlays] == [tree.joinpath(*segments)] + # The seed really reached the arm, so the base above was chosen with an + # overlay row in hand rather than for an empty spec list. + assert [spec.id for spec in wiring.overlays[0]["seeds"]] == ["overlay.demo"] + + +# -- lifecycle -------------------------------------------------------------- + + +async def test_core_lifespan_closes_the_collection_and_never_closes_a_worker( + tmp_path, monkeypatch +): + """``coll.close()`` stays in the core finally; no worker teardown here.""" + wiring = _wire( + monkeypatch, + harness=(_SOURCE,), + current=_SHA, + tree=_checkout(tmp_path), + catalog=_catalog(_provider_component()), + ) + stack = create_stack(config=_config(tmp_path)) + collection = wiring.collections[0] + async with stack._lifespan_manager(): + assert collection.starts == 1 + assert collection.closes == 0 + assert collection.closes == 1 + assert wiring.workers != [] + assert not hasattr(WorkerProvider, "close") + + +# -- frozen keyword surface (spec 14) --------------------------------------- + + +class TestCreateStackSignature: + """``create_stack``'s keyword surface is a published contract. + + Every host adapter, the CLI, and every embedder calls this by keyword. + A parameter renamed, reordered into a positional slot, or quietly added + breaks callers this repository cannot see, so the tuple is pinned rather + than described. ``create_plane`` grew ``extras``; ``create_stack`` did + not, and this is where that stays true. + """ + + #: Exactly today's parameters, in today's order. + PARAMETERS = ( + "collection", + "config", + "providers", + "disable", + "discover_entry_points", + "enable_path_safety", + "enable_response_limit", + "response_limit_bytes", + "validate_annotations", + "instructions", + ) + + def test_the_parameter_names_are_exactly_the_frozen_tuple(self): + assert tuple(inspect.signature(create_stack).parameters) == self.PARAMETERS + + def test_every_parameter_is_keyword_only(self): + parameters = inspect.signature(create_stack).parameters + kinds = {name: p.kind for name, p in parameters.items()} + assert kinds == dict.fromkeys(self.PARAMETERS, inspect.Parameter.KEYWORD_ONLY) diff --git a/tests/test_tool_hints.py b/tests/test_tool_hints.py index 1309d5c..aa4b79b 100644 --- a/tests/test_tool_hints.py +++ b/tests/test_tool_hints.py @@ -27,7 +27,8 @@ #: spelling the naming middleware rejects at registration time. _MOUNT_ERA = re.compile( r"\b(molcrafts|molvis|molq|molexp)_" - r"(packages|outline|open|compose|search|suggest|exec|close|refresh|" + r"(packages|outline|open|compose|search|suggest|list_planes|route|" + r"exec|close|refresh|" r"capabilities|poll_events|list_sessions|list_jobs|get_job|job_logs|" r"list_destinations|list_queue|submit_job|cancel_job|list_projects|" r"list_experiments|list_runs|workspace_layout|validate_workspace|" @@ -92,9 +93,16 @@ class TestSourceIsClean: "path", sorted(SRC.rglob("*.py")), ids=lambda p: str(p.name) ) def test_no_module_emits_a_mount_era_tool_name(self, path: Path): - # Two modules state the contract by quoting the spelling it bans; + # Modules that state the contract by quoting the spelling it bans; # for them the mount-era form appearing is the point. - if path.name in {"naming.py", "provider.py"} and path.parent.name in { + contract_files = { + "naming.py", + "planes.py", + "provider.py", + "provider_sdk.py", + "server.py", + } + if path.name in contract_files and path.parent.name in { "middleware", "molmcp", }: diff --git a/zensical.toml b/zensical.toml index 4833034..94e5e25 100644 --- a/zensical.toml +++ b/zensical.toml @@ -22,10 +22,13 @@ nav = [ { "Provider design" = "concepts/provider-design.md" }, { "Providers" = "concepts/providers.md" }, { "Middleware" = "concepts/middleware.md" }, + { "Harness catalog" = "concepts/harness.md" }, { "Expose a package" = "guides/expose-a-package.md" }, { "Write a Provider" = "guides/write-a-provider.md" }, { "MolVis workbench" = "guides/molvis-workbench.md" }, { "Adopt a data directory" = "guides/adopt-a-data-directory.md" }, + { "Iterate on a harness" = "guides/iterate-on-a-harness.md" }, + { "Harness migration" = "guides/harness-migration.md" }, { "Security" = "guides/security.md" }, ] }, { "Reference" = [