Prevent agents from calling duplicate tools with same inputs#9
Conversation
|
@wmeddie 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs. I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review. |
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
wmeddie
left a comment
There was a problem hiding this comment.
Thanks for the implementation. I found a couple of correctness issues that should be addressed before this is merged:
-
AgentLearnstill subclassesComponent, but itsexecute()now callsself.handle_tool_use(...).handle_tool_useis only defined onAgentRun, so any learning run that emits a tool call will raiseAttributeError: 'AgentLearn' object has no attribute 'handle_tool_use'instead of executing/preventing the tool call. Either makeAgentLearninherit/reuse theAgentRunimplementation or add the duplicate-prevention path toAgentLearndirectly. -
Plain-text tool arguments all normalize to
None, so different raw arguments for the same tool are treated as duplicates. For example,<tool name="lookup_memory">project timeline</tool>followed by<tool name="lookup_memory">project milestones</tool>is blocked even though the arguments differ.parse_tool_args()returns(None, raw_string)for non-JSON/non-XML args, but duplicate detection only stores/compares the parsed dict. The normalized key should include the raw argument string when parsing does not produce a dict.
I also noticed the new _duplicate_reasoning / _nonce metadata is left in tool_args_str when the tool is executed. That may be intentional for _nonce per the doc, but _duplicate_reasoning will be passed to standard tools/MCP tools and could break strict schemas; consider stripping framework-only metadata before dispatch if tools are not expected to receive it.
Validation run locally:
python -m py_compile agent_components.pypassed.python -m pytest -qfound no tests.
This PR implements duplicate tool call prevention to address a common failure mode where frontier models (GPT-5, Gemini-Flash-2.5) get stuck calling the same tool repeatedly with identical inputs.
Problem
Agents would sometimes enter infinite loops calling tools like:
Solution
The implementation tracks tool calls within each
AgentRunexecution and blocks duplicates based on tool name + normalized arguments. However, agents can override this by providing unique reasoning:Key Features
_nonceand_duplicate_reasoningfrom duplicate detection_duplicate_reasoningis unique within the current sessionComponents Affected
AgentRun: Primary implementation with tool call history trackingAgentLearn: Extended to support duplicate preventionself.tool_call_history = []and reset it on each executionTechnical Implementation
Added helper functions:
normalize_tool_args(): Removes special parameters for comparisonextract_nonce_and_reasoning(): Extracts special parameters from parsed argsis_duplicate_tool_call(): Checks if tool call matches previous callsis_reasoning_unique(): Validates reasoning hasn't been used beforeThe duplicate check is integrated into
handle_tool_use()before tool execution, providing immediate feedback when duplicates are detected.Fixes #8.
💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.