The original v2.0 idea of MultiNet is no longer at the frontier due to the pace at which model capabilities are improving. Redesign what the benchmark will be such that it will be at the frontier in the next 3-4 months, but at the same time maintain the thesis and evaluate primarily cross-domain generalization.
Outcome - a concrete v2.0 spec that the team can orient around and build out
The original v2.0 idea of MultiNet is no longer at the frontier due to the pace at which model capabilities are improving. Redesign what the benchmark will be such that it will be at the frontier in the next 3-4 months, but at the same time maintain the thesis and evaluate primarily cross-domain generalization.
Outcome - a concrete v2.0 spec that the team can orient around and build out