Skip to content

fix(langchain): preserve tool calls in generation traces - #3038

Open
Kuang-xianxin wants to merge 1 commit into
Chainlit:mainfrom
Kuang-xianxin:codex/preserve-langchain-tool-call-traces
Open

Kuang-xianxin wants to merge 1 commit into
Chainlit:mainfrom
Kuang-xianxin:codex/preserve-langchain-tool-call-traces

Conversation

@Kuang-xianxin

@Kuang-xianxin Kuang-xianxin commented Sep 5, 2026

Copy link
Copy Markdown

Problem and change

LangChain exposes provider-independent tool calls in AIMessage.tool_calls and correlates results through ToolMessage.tool_call_id. Chainlit's generation converter drops both: a message such as AIMessage(content="", tool_calls=[{"name": "weather", "args": {"city": "Paris"}, "id": "call_1"}]) is recorded as an empty assistant message, and its tool result loses call_1.

Preserve normalized call IDs, names and arguments, and the tool-result ID, for both live messages and LangChain's serialized constructor dictionaries. Prefer normalized calls when a provider also puts the same calls in content blocks, so Anthropic calls are not duplicated. Existing text content, legacy function_call, and content-only Anthropic tool_use handling are retained. This repairs generation tracing; it does not change agent tool execution.

Validation

  • Before the fix: six new conversion cases and the real async callback-manager regression fail on missing tool calls or result correlation.
  • After the fix: 76 tests pass with PYTHONPATH=backend uv run --no-sync pytest backend/tests/langchain backend/tests/test_step.py -q (Python 3.13.14, LangChain Core 1.2.27, LiteralAI 0.1.201).
  • Added compatibility controls for legacy function calls and Anthropic content with/without normalized calls. The async test drives AsyncCallbackManager through start/end events and inspects the resulting generation payload, without a model API request.
  • Project Python lint/format wrappers pass for the three changed files; scoped source mypy passes.
  • Full backend mypy was attempted. Optional integration dependencies/stubs are missing in the local environment, so it is not reported as passing. Full-extra installation is blocked by the lock's NumPy 1.26.4 build on Python 3.13. Frontend/E2E and live-provider suites were not run.

Found through source inspection; searches for LangChain/tool-call PRs did not identify an overlapping open fix. No existing issue is claimed as resolved.

No manual browser verification was performed.


Summary by cubic

Fixes LangChain generation tracing so tool calls and tool-result IDs are preserved in the persisted step.

  • Records normalized tool calls (name, args, id) from AIMessage.tool_calls in both live and serialized message conversion.
  • Records tool_call_id on tool result messages so calls and results stay correlated.
  • Prefers normalized tool calls over provider-specific content blocks, preventing duplicate Anthropic tool_use entries.
  • Retains existing text content, legacy function_call, and content-only Anthropic handling.

Written for commit 45bae9d. Summary will update on new commits.

Review in cubic

Keep normalized tool calls and tool-result IDs in both message conversion paths.
Cover real async callback events and legacy provider message formats.

Co-Authored-By: Codex <noreply@openai.com>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 3 files

Re-trigger cubic

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant