Skip to content

feat(stores): local-fs and user-local outside-repo backends (feature 034) - #412

Merged
Marc-cn merged 2 commits into
darnitdevorg:mainfrom
mlieberman85:034-local-output-store
Sep 2, 2026
Merged

feat(stores): local-fs and user-local outside-repo backends (feature 034)#412
Marc-cn merged 2 commits into
darnitdevorg:mainfrom
mlieberman85:034-local-output-store

Conversation

@mlieberman85

Copy link
Copy Markdown
Contributor

Summary

Two new filesystem-backed Store backends inside darnit-core that write outside the audited repository:

  • local-fs: config-driven root (absolute, ~-expanded, or $VAR-templated). Fail-fast on missing env vars.
  • user-local: platform-conventional root (XDG on Linux, Apple support/cache dirs on macOS, %LOCALAPPDATA% on Windows). No root config needed.

Both are selectable via .baseline.toml:

```toml
[stores.attestation]
backend = "local-fs"
root = "$DARNIT_ATT_ROOT"

[stores.report]
backend = "user-local"
```

Motivation

Feature 033 (#396) shipped the pluggable-stores abstraction. Its filesystem defaults all write inside the audited repo (<repo>/.darnit/attestations/, etc.). This PR closes the "put it somewhere else on the local filesystem" gap that OSPO leaders keep hitting (consolidate attestations across 30 repos into one directory) and that CI operators keep hitting (cache in the runner cache dir, reports in the runner artifacts dir).

Motivating scenarios (in spec.md):

  1. OSPO leader consolidates every audit's attestation into ~/darnit-attestations/.
  2. CI operator points cache at $RUNNER_CACHE_DIR/darnit and reports at $RUNNER_ARTIFACTS_DIR/darnit-reports.
  3. Individual dev uses backend = "user-local" and lets XDG / macOS conventions decide.

Multi-repo templating: env-var interpolation is the escape hatch. root = \"\$DARNIT_ATT_ROOT\" in every repo's .baseline.toml plus one per-machine env var. No new config layer (spec Q1 clarify) -- this stays within .baseline.toml.

Design highlights

  • No new runtime dependencies (FR-014). platform_paths.py hand-rolls XDG / Apple / Windows conventions in ~90 lines rather than pulling in platformdirs.
  • No new Store Protocol methods. Both backends satisfy the feature-033 Protocols by delegating to the in-repo classes after root resolution.
  • .project/project.yaml stays in the repo (FR-009). Neither backend is registered under darnit.stores.project; [stores.project] backend = "user-local" raises `StoreNotInstalled` at `resolve_stores` time before any control runs.
  • Path traversal impossible by construction (SC-005). Shared _sanitize_filename regex from feature 033 handles it.
  • Cross-filesystem cache writes safe (research R-004). The delegate already writes the tempfile into the target's directory.
  • Feature 033's zero-config test passes unchanged (SC-003). New test_us4_zero_config_local.py extends the invariant to the new backends.
  • Loud error surface (SC-008). LocalFs*/UserLocal* wrap the delegate's `StoreOperationError` with a message that names the backend, kind, and resolved path.
  • Info-log on every outside-repo write (FR-015). darnit.stores.local logger emits wrote <kind> (<backend>): <resolved-path> per artifact. Pre-feature filesystem defaults emit zero lines to that logger.

Constitution check

All five principles PASS (from plan.md):

Principle Status
I. Plugin Separation PASS (new code in darnit-core; no cross-package imports added)
II. Conservative-by-Default PASS (no silent fallback; per-Protocol failure semantics preserved)
III. TOML-First Architecture PASS (config surface stays TOML)
IV. Never Guess User Values N/A (storage backends don't produce user-judgment values)
V. Sieve Pipeline Integrity N/A (backends are not sieve passes)

Deferrals (documented in tasks.md)

  • T015: LocalFsProjectStateStore deferred. Would require reimplementing .project/-prefix-free YAML I/O; better handled in a separate PR alongside the audit driver's use of bundle.project.
  • T041: Windows CI runner integration test. Deferred because no Windows CI runner is currently configured. Windows path resolution is unit-tested via platform.system() monkeypatch, so SC-004 is satisfied at the unit level.

Spec artifacts

Under specs/034-local-output-store/:

  • spec.md: 4 user stories, 15 FRs, 9 SCs, 3-question clarify session
  • plan.md, research.md, data-model.md, contracts/, quickstart.md
  • tasks.md: 41 tasks in 7 phases, 39/41 completed (T015, T041 deferred with rationale)
  • checklists/requirements.md: 16/16 pass

Test plan

  • Baseline before change: 97 stores tests pass.
  • After Phase 3 (US1 MVP): 120 pass.
  • After Phase 4 (US2): 135 pass.
  • After Phase 5 (US3): full stores suite 157 pass.
  • Feature 033's test_us2_zero_config.py passes unchanged (T032).
  • Full workspace sweep: 3061 pass, 26 skip, 0 fail.
  • ruff check . clean repo-wide.
  • validate_sync.py clean.
  • Structure guard: all changes within stores/defaults/, pyproject.toml, stores/__init__.py, docs/plugin-authoring/stores.md, tests/darnit/stores/ (plus speckit bookkeeping).
  • FR-014 no-new-deps guard: pyproject.toml diff is entry-point registrations only.

Files changed

Production code (new):

  • packages/darnit/src/darnit/stores/defaults/local_fs.py (~200 lines)
  • packages/darnit/src/darnit/stores/defaults/user_local.py (~120 lines)
  • packages/darnit/src/darnit/stores/defaults/platform_paths.py (~90 lines)

Production code (modified):

  • packages/darnit/src/darnit/stores/defaults/__init__.py (re-exports)
  • packages/darnit/pyproject.toml (6 entry-point registrations)

Docs:

  • docs/plugin-authoring/stores.md ("Writing artifacts outside the repo" section)

Tests (new):

  • tests/darnit/stores/test_local_fs_backend.py
  • tests/darnit/stores/test_local_fs_helpers.py
  • tests/darnit/stores/test_local_fs_isolation.py
  • tests/darnit/stores/test_local_fs_logging.py
  • tests/darnit/stores/test_platform_paths.py
  • tests/darnit/stores/test_us4_zero_config_local.py
  • tests/darnit/stores/test_user_local_backend.py

Speckit artifacts: specs/034-local-output-store/ + speckit-bookkeeping updates to .specify/feature.json and CLAUDE.md.

…evorg#356)

A new user opens `darnit --help` and sees `audit`, `run`, `serve`,
`harness`, `install` at the same visual weight. Nothing tells them
which is the front door. The intended product path -- skills invoking
darnit's MCP tools inside a coding-agent client -- lives in
maintainer heads and RFC-0001 today, not in a place a user would
find it.

Adds a "How to Use Darnit" section to README.md between Installation
and Quick Start:

1. Product path: install the MCP config + skills into Claude Code /
   Claude Desktop / Cursor via `darnit install`, then invoke a skill
   (e.g. `/darnit-audit`) in the agent. The skill orchestrates the
   MCP tools; the user reasons conversationally.
2. CLI as dev/test scaffolding: a per-command table names what each
   of `serve`, `audit`, `run`, `harness` is for and what it is NOT
   for. Only `serve` is product-facing (the MCP server behind the
   skills). `audit`/`run`/`harness` are development, CI, and driver-
   testing tools respectively.
3. Direction of travel: cross-links RFC-0001, which formalizes the
   split (CLI becomes thin adapters around the harness runtime, not
   parallel entry points).

Also re-frames the existing Quick Start intro: the Python code
snippets shown are MCP tool signatures the `/darnit-audit` skill
calls under the hood; in normal use the reader invokes the skill
and never sees them directly. Preserves the snippets for the
"embed darnit in your own tooling" and "debug the skill's
orchestration" cases.

`docs/getting-started/README.md` already has an "I want to use
darnit with Claude Code" path -- unchanged; that section is
consistent with the new README framing.
…034)

Complements feature 033 (darnitdevorg#396) with two new filesystem-backed
`Store` backends inside darnit-core that write outside the audited
repository:

* `local-fs`: takes a config-driven `root` (absolute, ~-expanded, or
  $VAR-templated). $VAR interpolation uses `missing="raise"` mode so a
  typo is a hard KeyError at store construction, not a silent empty
  expansion.
* `user-local`: resolves `root` from platform conventions (XDG on
  Linux, ~/Library/{Application Support,Caches}/darnit on macOS,
  %LOCALAPPDATA%\darnit\{Data,Cache}\ on Windows). No `root` config
  needed; explicit `root` kwargs are warn-and-ignored.

Both selectable via `.baseline.toml`:

  [stores.attestation]
  backend = "local-fs"
  root    = "$DARNIT_ATT_ROOT"

  [stores.report]
  backend = "user-local"

The OSPO-leader consolidation use case (US1): env-var interpolation
lets 30 repos share `root = "$DARNIT_ATT_ROOT"` in their
.baseline.toml files with one per-machine env var doing the routing.
No new config layer needed.

Every successful outside-repo write emits one info-level log line to
`darnit.stores.local` naming the backend, artifact kind, and resolved
absolute path (FR-015 / SC-009). The pre-feature in-repo defaults
emit zero lines to that logger, so zero-config audits stay
log-silent.

Design highlights:

* No new runtime dependencies (FR-014). platform_paths.py hand-rolls
  XDG / macOS / Windows conventions in ~90 lines rather than pulling
  in `platformdirs`.
* No `Store` Protocol methods added -- both backends satisfy the
  existing feature-033 Protocols by delegating to the in-repo classes
  after root resolution.
* `.project/project.yaml` stays in the repo (FR-009). Neither backend
  is registered under `darnit.stores.project`; `[stores.project]
  backend = "user-local"` raises `StoreNotInstalled` at
  `resolve_stores` time before any control runs.
* Path traversal is impossible by construction (SC-005). The shared
  `_sanitize_filename` regex from feature 033 handles it; a
  bundle_id like `"../../etc/foo"` produces a filename inside `root`,
  not a path escape.
* Cross-filesystem cache writes are safe (R-004). The delegate
  already writes the tempfile into the target's directory, so
  os.replace never crosses filesystems.
* Feature 033's US2 zero-config test passes unchanged (SC-003). New
  test_us4_zero_config_local.py extends the invariant to the new
  backends.
* Two-step SC-008 error surface: LocalFs*/UserLocal* wrap the
  delegate's StoreOperationError with a message that names the
  backend, kind, and resolved path so operators can correlate the
  failure to their config.

Spec artifacts under specs/034-local-output-store/:
* spec.md: 4 user stories, 15 FRs, 9 SCs, 3-question clarify session
* plan.md: constitution check PASS, project structure decision
* research.md: 5 resolved research items (R-001 through R-005)
* data-model.md: 5 entities including the info-log format
* contracts/local-fs.md and contracts/user-local.md
* quickstart.md: 3 worked examples
* tasks.md: 41 tasks in 7 phases, all completed except T015 (deferred
  to a future PR because reimplementing .project/-prefix-free YAML
  I/O was scope creep) and T041 (Windows CI stretch goal)

Test totals: baseline 97 stores tests -> 135 (+38 new). Full
workspace sweep: 3061 pass, 26 skip, 0 fail. Ruff clean repo-wide.
validate_sync clean. Structure and no-new-deps guards pass.
@Marc-cn

Marc-cn commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Tested on Linux. Stores suite 157/157. Both backends behave as described. Nothing landed inside the audited repo.
Zero-config check: audit on tqdm is byte-identical to main across all 66 controls, so SC-003 holds. lgtm

@Marc-cn
Marc-cn merged commit 73387b5 into darnitdevorg:main Sep 2, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants