Skip to content

Do not use the final result count as the retrieval candidate limit #88

Description

@tulayha

When a caller asks VidXP for three final results, the same limit is currently used while reading candidates from each underlying search index.

That can hide a strong combined result before ranking happens. For example, a moment ranked fourth in dialogue and fourth in visual search may become the best result when both signals are combined, but VidXP never sees it if each search returns only three candidates.

What should change:

  • Use a separate internal candidate limit for each search channel.
  • Keep that internal pool bounded so searches cannot grow without limit.
  • Continue using top_k only for the number of final results returned to the caller.
  • Preserve the existing CLI, API, and MCP meaning of top_k.
  • Add a regression test where the best combined result falls outside the initial per-channel top_k.
  • Use Add end-to-end retrieval evaluation #76 to measure the recall improvement and added search cost before choosing the default candidate depth.

Metadata

Metadata

Assignees

Labels

help wantedExtra attention is needed

Type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions