Skip to content

Prevent agents from calling duplicate tools with same inputs - #9

Draft
wmeddie with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-8
Draft

Prevent agents from calling duplicate tools with same inputs#9
wmeddie with Copilot wants to merge 4 commits into
mainfrom
copilot/fix-8

Conversation

Copilot AI commented Aug 11, 2025

Copy link
Copy Markdown

This PR implements duplicate tool call prevention to address a common failure mode where frontier models (GPT-5, Gemini-Flash-2.5) get stuck calling the same tool repeatedly with identical inputs.

Problem

Agents would sometimes enter infinite loops calling tools like:

<tool name="lookup_memory">{"query": "project timeline"}</tool>
<tool name="lookup_memory">{"query": "project timeline"}</tool>
<tool name="lookup_memory">{"query": "project timeline"}</tool>
<!-- ... continues indefinitely -->

Solution

The implementation tracks tool calls within each AgentRun execution and blocks duplicates based on tool name + normalized arguments. However, agents can override this by providing unique reasoning:

<!-- First call - allowed -->
<tool name="lookup_memory">{"query": "project timeline"}</tool>

<!-- Exact duplicate - blocked -->
<tool name="lookup_memory">{"query": "project timeline"}</tool>

<!-- Duplicate with unique reasoning - allowed -->
<tool name="lookup_memory">{"query": "project timeline", "_duplicate_reasoning": "searching with broader context"}</tool>

Key Features

  • Argument normalization: Excludes _nonce and _duplicate_reasoning from duplicate detection
  • Reasoning validation: Ensures _duplicate_reasoning is unique within the current session
  • Format support: Works with both JSON and XML argument formats
  • Clear error messages: Guides agents when duplicates are blocked
  • Whitespace handling: Normalizes reasoning to prevent trivial bypasses
  • Stress level adjustment: Increases when duplicates are blocked to encourage different behavior

Components Affected

  • AgentRun: Primary implementation with tool call history tracking
  • AgentLearn: Extended to support duplicate prevention
  • Both components now initialize self.tool_call_history = [] and reset it on each execution

Technical Implementation

Added helper functions:

  • normalize_tool_args(): Removes special parameters for comparison
  • extract_nonce_and_reasoning(): Extracts special parameters from parsed args
  • is_duplicate_tool_call(): Checks if tool call matches previous calls
  • is_reasoning_unique(): Validates reasoning hasn't been used before

The duplicate check is integrated into handle_tool_use() before tool execution, providing immediate feedback when duplicates are detected.

Fixes #8.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI self-assigned this Aug 11, 2025

Copilot AI commented Aug 11, 2025

Copy link
Copy Markdown
Author

@wmeddie 👋 This repository doesn't have Copilot instructions. With Copilot instructions, I can understand the repository better, work faster and produce higher quality PRs.

I can generate a .github/copilot-instructions.md file for you automatically. Click here to open a pre-filled issue and assign it to me. I'll write the instructions, and then tag you for review.

Copilot AI and others added 3 commits August 11, 2025 18:08
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
Co-authored-by: wmeddie <122480+wmeddie@users.noreply.github.com>
Copilot AI changed the title [WIP] We need to prevent agents to call the same tool multiple times in a row with the same inputs. Prevent agents from calling duplicate tools with same inputs Aug 11, 2025
Copilot AI requested a review from wmeddie August 11, 2025 18:16

@wmeddie wmeddie left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the implementation. I found a couple of correctness issues that should be addressed before this is merged:

  1. AgentLearn still subclasses Component, but its execute() now calls self.handle_tool_use(...). handle_tool_use is only defined on AgentRun, so any learning run that emits a tool call will raise AttributeError: 'AgentLearn' object has no attribute 'handle_tool_use' instead of executing/preventing the tool call. Either make AgentLearn inherit/reuse the AgentRun implementation or add the duplicate-prevention path to AgentLearn directly.

  2. Plain-text tool arguments all normalize to None, so different raw arguments for the same tool are treated as duplicates. For example, <tool name="lookup_memory">project timeline</tool> followed by <tool name="lookup_memory">project milestones</tool> is blocked even though the arguments differ. parse_tool_args() returns (None, raw_string) for non-JSON/non-XML args, but duplicate detection only stores/compares the parsed dict. The normalized key should include the raw argument string when parsing does not produce a dict.

I also noticed the new _duplicate_reasoning / _nonce metadata is left in tool_args_str when the tool is executed. That may be intentional for _nonce per the doc, but _duplicate_reasoning will be passed to standard tools/MCP tools and could break strict schemas; consider stripping framework-only metadata before dispatch if tools are not expected to receive it.

Validation run locally:

  • python -m py_compile agent_components.py passed.
  • python -m pytest -q found no tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

We need to prevent agents to call the same tool multiple times in a row with the same inputs.

2 participants