Skip to content

Preserve tracker tags when an execution completes - #37

Merged
adham90 merged 1 commit into
adham90:mainfrom
symphony-stream:fix/preserve-tracker-tags-on-completion
Jul 30, 2026
Merged

Preserve tracker tags when an execution completes#37
adham90 merged 1 commit into
adham90:mainfrom
symphony-stream:fix/preserve-tracker-tags-on-completion

Conversation

@symphony-stream

Copy link
Copy Markdown
Contributor

Defect

Instrumentation#build_running_execution_data injects the active tracker's request_id and tags (via inject_tracker_data) into the execution's metadata when the "running" row is created. But when the run completes, build_completion_data rebuilds metadata from agent metadata + context metadata and never carries the tracker fields over. So whenever the completion rebuild produces any metadata at all — which it does on essentially every successful LLM run, since request timing/count land in context metadata — the update! replaces the whole metadata column and the tags correlation written at start is gone.

Error runs keep their tags only by accident: their rebuild usually produces no metadata, so the column is left untouched.

Reproduction

middleware = RubyLLM::Agents::Pipeline::Middleware::Instrumentation.new(app, MyAgent)

RubyLLM::Agents.track(tags: {feature: "voice-chat", session_id: "sess_1"}) do
  middleware.call(context) # successful run that records llm_request_count in metadata
end

execution = RubyLLM::Agents::Execution.last
execution.metadata["llm_request_count"] # => 1
execution.metadata["tags"]              # => nil (expected: {"feature" => "voice-chat", "session_id" => "sess_1"})

Fix

At the end of build_completion_data, call the existing inject_tracker_data(context, data) — the same helper build_running_execution_data uses — but only when the rebuild actually produced metadata (if data[:metadata]). The guard keeps the no-metadata case on the previous behavior: data carries no metadata key, the update! leaves the column alone, and the running record's metadata (tags included) is preserved. This also avoids replacing the column with a tags-only hash, so other keys written at start (e.g. replay_source_id) can never be clobbered by the completion.

Tests

New tracker tags on completion examples in spec/lib/pipeline/middleware/instrumentation_spec.rb:

  • successful run keeps tracker tags alongside rebuilt metadata (fails on main — tags are dropped)
  • failed executions keep tracker tags (pins the existing behavior)
  • no tags are added when no tracker was active
  • an empty rebuild leaves completion metadata unset, preserving the running record's metadata

Full suite: 4900 examples, 0 failures (Ruby 3.3.6). standardrb clean on changed files.

build_running_execution_data injects the active tracker's request_id and
tags into execution metadata, but build_completion_data rebuilds metadata
from agent + context metadata and drops them. Any completion that
produces metadata (every successful LLM run records request timing) then
replaces the column and the tags correlation is lost; error runs keep it
only because their rebuild usually produces no metadata.

Carry the tracker fields through the completion rebuild via the existing
inject_tracker_data helper, but only when the rebuild actually produced
metadata — when it did not, leaving the column untouched preserves the
running record's metadata as before.
@adham90
adham90 merged commit 68bfcc5 into adham90:main Jul 30, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants