Skip to content

History: the operation journal and the per-slot archive (foundation) #72

Description

@AliceLafox

Every mutation already leaves a safety net behind: a full backup of ROLAND/DATA
into the app's backups/<stamp>/, and, for anything that rewrites or removes
audio, a copy of the original WAV in trash/<stamp>/<NNN_1>/. What is missing is
the record. operations.log is one human-readable line per operation, written
only by push and normalize, and nothing machine-readable ties a slot to what used
to be in it.

This is the foundation of the history feature. No UI lands here.

The bug it fixes first

The stamp is wall clock at one second resolution, taken when the job is enqueued,
and it names both the backup and the trash directory. Operations that share a
second share the directory, and the backup overwrites: a bulk normalize of slots
32, 33 and 34 produced a single backup directory for three operations, so two
of the three pre-states are gone. The audio survived only because the three slots
have different directories inside the stamp — a batch that touches one slot twice
loses the earlier take outright.

Operation identity becomes a unique id. The wall clock stays as display metadata,
where it cannot lose anything.

What lands

  • Two stores in the app tier. The core stays header-only and dependency-free:
    commands return facts, the app records them. history.db on WAL holds cards,
    sessions, ops, slot_states, slot_head, blobs_meta. audio.db holds
    blobs(hash PRIMARY KEY, bytes BLOB) on journal_mode=DELETE, page_size=16384,
    and auto_vacuum=INCREMENTAL set before its first table — that mode cannot be
    turned on later without rewriting the file, and retention (History storage: show what it costs, free it on request #74) needs it.
  • Per-slot records. For each touched slot, the slot body verbatim — 1142
    bytes, median, against 228 KB for a whole-DATA backup — plus a manifest of its
    audio: names, sizes, content hash. slot_states(card, slot, op DESC) is the
    slot's timeline; the same rows without the slot key are the global one.
  • Audio content-addressed in a database, not in a folder. The hash costs nothing:
    the archive copy already reads the whole file into RAM. It buys refcounting and
    integrity rather than space — measured on a development machine, 47 objects carry 40
    distinct hashes, so deduplication saves 74 MB of 1534, or 4.8%. Loops are mostly not
    each other.
  • One transaction over both stores. ATTACH, and the row and its bytes commit
    together. There is no write-ordering rule to remember in every new code path, and a row
    pointing at absent bytes cannot happen by construction. Measured on our real sizes (30
    takes, 667 MB): blob writes in DELETE mode land in 0.52-0.97 s against 0.16-0.76 s for
    plain files, with 0.2 MB of overhead on 667 MB. WAL was the trap for audio — a permanent
    +24% on disk and a VACUUM that does not give the space back.
  • A baseline at connect. The pedal is the other writer: it records loops
    while the app is not looking, and in STORAGE mode it is not a looper, so drift
    happens only between sessions. On connect, compare the card against what we
    left — slot body hashes from the document the scan already reads, WAV names and
    sizes from the listing it already walks — and append a row per slot that moved.
    Full audio hashing does not belong here: it belongs to the moment of restore,
    and to an explicit Maintenance check. An overdub of identical length can hide
    from the cheap signals; it cannot be lost, because a restore archives what it
    is about to overwrite and hashes it on the way.
  • Legacy backfill. The existing backups/ and trash/ directories import as
    rows marked legacy, with their uncertain provenance stated rather than guessed.
    Nothing is deleted.

Tests, from the invariants

  • Two operations in the same second keep both pre-states.
  • A process killed mid-write leaves a store that opens, with no half-written row in it.
  • The same audio pushed to two slots stores one object.
  • A card that changed while unplugged yields one row per moved slot and none for
    the slots that stand still.
  • Legacy directories import without inventing a provenance they do not have.
  • A slot body restored from a record is byte-identical to the one recorded.

The slices this carries

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions