Skip to content

Space discovery: replace the [0-9]-*/ glob with a marker file #41

Description

@plur9

Spaces are discovered by globbing [0-9]-*/ at the install root. That pattern is
duplicated across 15+ call sites with no shared helper, and it conflates three
different things: whether a directory is a space, what kind of space it is,
and its local sort order.

The duplication

Fifteen independent call sites, each rolling its own glob against a differently
named base:

ROOT.glob("[0-9]-*")                             migrate_org_tag.py:40
DATA_DIR.glob('[0-9]-*')                         install_space_hooks.py:32
DATA_DIR.glob('[0-9]-*')                         gtd_hygiene.py:92
DATACORE_ROOT.glob("[0-9]-*/.datacore/agents")   add_engram_preamble.py:47
root.glob("[0-9]-*/org/intents.org")             priority_score.py:132
root.glob("[0-9]-*/.datacore/tags.yaml")         priority_score.py:246
data_dir.glob("[0-9]-*/org")                     stamp_seq_todo.py:67
data_root.glob('[0-9]-*')                        tag_utils.py:76
root.glob("[0-9]-*/**/*.org")                    org_tag_audit.py:87
data_dir.glob("[0-9]-*/org/*.org")               zombie_disposition_2026_07_25.py:34
…

plus five parallel "discover spaces" implementations: agent_wrap_up.discover_spaces,
hooks._hook_discover_spaces, nightshift_inbox_cleanup.discover_spaces,
nightshift_archival.find_all_spaces, and private ones inside tag_validator
and zettel_db.

Adding a space type, or changing where spaces may live, currently means a
15-file sweep. Miss one and that space silently disappears from that subsystem —
no exception, just absent from journals, or hygiene runs, or tag validation,
until someone notices weeks later.

Three problems with the pattern itself

The numeric prefix is local, not identity. Each install numbers its own
spaces; the same space repo is 5-x in one install and 9-x in another. So the
prefix is a sort key being used as an identity marker, and nothing about it is
portable.

It cannot express kind. Every space is treated identically because the glob
is all the machinery has. There is no way for a subsystem to say "process team
spaces but not this other kind."

It over- and under-matches. 0-inbox is an install-level capture point, not
a team space, but matches the glob and is processed as one. Conversely a space
cannot live anywhere but the install root, even when the owning relationship
would put it elsewhere.

Proposal: discover by marker

A directory is a space if it contains .datacore/config.yaml with a space:
key — the same way git finds .git and monorepo tools find package.json.

space:
  name: example
  type: team          # personal | team | meta | client
  owner: some-space   # optional — the space that owns this one
  • Location stops mattering. Root, nested, anywhere.
  • The numeric prefix becomes what it always was: local cosmetics.
  • type lets subsystems opt in or out by kind.
  • owner records a relationship without encoding it as a directory, so it
    survives the relationship changing.
  • Nothing to rot, provided the scaffolder writes the marker (see below).

Current coverage

glob    [0-9]-*/                        9 of 9   ← only complete mechanism
marker  .datacore/config.yaml space:    5 of 9
registry install.yaml spaces:           2 of 9

install.yaml already has a spaces: registry with explicit path: entries and
is at 2/9. It decayed precisely because glob discovery made registration
optional. Any mechanism that asks a human to register a space will reach the
same state.
The marker must be emitted by create-space, and
scaffolding-auditor / structural-integrity must flag a space directory
without one.

Migration, without silent breakage

The failure mode to design against is a space vanishing quietly. So:

  1. Add discover_spaces() returning (path, config) pairs, with a bounded walk
    and a skip list (.git, node_modules, 2-projects, 4-archive).
  2. Run dual mode: return the union of glob results and marker results, and
    log any set difference at debug level.
  3. Backfill markers for the spaces missing them.
  4. Migrate the 15 call sites onto the helper.
  5. Add a test asserting the glob set and marker set are identical.
  6. Only once that test is green, drop the glob.

Steps 1–4 change no behaviour: the union is a superset of today's result.

Note for anyone with an existing space clone

A space repo that does not commit .datacore/config.yaml will keep working
through dual mode, but must gain a marker before the glob is dropped. Worth
auditing space repos for this rather than discovering it at removal time.

Scope

  • discover_spaces() helper + tests
  • marker backfill for unmarked spaces
  • migrate the 15 call sites and retire the five parallel implementations
  • create-space emits the marker; scaffolding audit flags its absence
  • extend DIP-0015 (Semantic Organization) to specify space identity and
    discovery — it owns folder conventions and structural-integrity

Dropping the glob is deliberately not in scope for the first PR.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions