Track source turn/window provenance for persisted world state

[chiooone]

Motivation

Before encountering this part of Codex, I had already been experimenting with a similar state-management pattern in my own projects. Seeing Codex use a related approach was genuinely exciting, and it made me pay closer attention to how persisted state evolves across turns and context windows.

That is what led me to notice that historical world state preserves what the state is, but not explicitly where that state came from. A persisted conclusion can therefore lose the context in which it was produced and later look more timeless or authoritative than it really is.

I think retaining source provenance would give future reconciliation logic a useful foundation: an older world-state conclusion could be distinguished from newer evidence instead of being treated as context-free truth. This change does not by itself implement conflict resolution or self-correction; it provides the provenance needed to make those behaviors more reliable later.

Proposal

Add optional source provenance to WorldStateItem:

• source_turn_id - the turn that produced the persisted state when the write is turn-scoped
• source_window_id - the context window that produced the persisted state

Populate these fields when world state is persisted during normal session updates, and preserve the corresponding provenance when world state is materialized during compaction.

The fields remain optional and are omitted when absent, so existing rollouts continue to deserialize and serialize compatibly.

Working implementation

I prepared a small implementation in my fork because opening pull requests to openai/codex is currently restricted to repository collaborators.

• Fork: https://github.com/chiooone/codex
• Branch: https://github.com/chiooone/codex/tree/feat/world-state-source-provenance
• Commit: chiooone@9dd7ab6

The prototype changes only three files and adds tests covering both persisted provenance and legacy world-state deserialization.

Validation

The implementation was checked with:

• just test -p codex-core world_state_rollout_persists_source_provenance
• just test -p codex-core legacy_world_state_item_deserializes_without_provenance
• just fix -p codex-protocol
• just fix -p codex-core
• just fmt
• git diff --check

Both new codex-core tests pass. The existing codex-protocol test effective_workspace_intersection_preserves_network_metadata_and_temp still fails on unmodified HEAD as well, so that baseline failure appears unrelated to this change.

I would be interested in feedback on whether this kind of provenance belongs directly on WorldStateItem, and how maintainers would prefer it to participate in future state reconciliation across turns and context windows.

[ArcticFox2029]

Worth separating two things that both get called provenance here, because they need different fields and only one of them is cheap.

Where a conclusion came from — turn, window, the tool output it was drawn from. That is what you're describing, and it is a pointer. Cheap to write, cheap to store, and it lets a later turn ask "was this before or after the thing I just learned".

Whether the world it described still exists. That one is not a pointer, and it is the failure I'd worry about more. A persisted conclusion doesn't usually go wrong by being old; it goes wrong by being about a file that has since been deleted, a function that has been renamed, a service that moved. Turn provenance tells you the conclusion is from turn 14. It doesn't tell you turn 14's world is gone.

I hit the second one hard enough to have a scar. A persisted index named seven files, every one of them deleted, and the staleness check reported zero seconds behind — because deleting a file moves no modification time forward, so "how old is this" was the wrong question and it was the only one being asked. The state was internally consistent, correctly dated, and entirely fictional. Age was not the missing field; reachability was.

So if you do add turn provenance, the cheap thing to add beside it is whatever identifies the world the conclusion is about — the paths it named, a content hash, the symbol names. Then reconciliation has something to check against instead of a timestamp to compare, and a conclusion can be marked unverifiable rather than silently outranked by recency.

None of that argues against your proposal. It's that turn provenance makes a conclusion comparable, and the thing above makes it falsifiable, and I'd want both before letting reconciliation logic pick a winner.

[chiooone]

Thanks — this is a really useful distinction.

You're right that I was using “provenance” to describe the cheap pointer-like part: where a conclusion was produced, so later reconciliation can place it relative to newer evidence. That makes a conclusion comparable, but not necessarily valid.

The reachability point gets at the more important failure mode: a state item can be perfectly well dated and still describe a world that no longer exists. Recording the relevant paths, content hashes, or symbols would give reconciliation something concrete to re-check, and “unverifiable” is a much better outcome than silently treating recency as correctness.

I still think turn/window provenance is a useful small, backwards-compatible first step, but I agree that a robust reconciliation design should eventually include both layers: provenance for comparability, and world references for falsifiability.

Your deleted-index example makes the distinction especially clear. Thanks for bringing the practical failure case into the discussion.