Skip to content

Report queued-vs-requested counts in the cassette/MockLLM exhaustion error #3

Description

@simukappu

When an agent makes more LLM calls than the cassette (or MockLLM) has responses for, agentverify raises CassetteMissingRequestError. For newcomers the failure is hard to read because the message does not say how far the run got. Adding the counts ("the agent made request N, but only M responses were recorded/queued") makes the cause obvious.

Acceptance criteria

  • The CassetteMissingRequestError message includes the index of the request that ran out and the total number of responses available.
  • Wording works for both the cassette-replay path and the MockLLM path (or each path passes its own count).
  • A test asserts the new message contains both numbers.
  • No change to when the error is raised, only to the message.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions