Backtest/sweep result metrics look wrong: CAGR overflow, maxDD% > 100%, win-rate/PnL inconsistency

[leonardojgv]

Backtest/sweep result metrics look wrong: CAGR overflow, maxDD% > 100%, win-rate/PnL inconsistency

Found while retesting #3 on 0.10.3, after backtests/sweeps started completing successfully again.
These are about the values returned for a COMPLETED run, not about jobs failing or hanging.

Strategy used throughout: a plain 9/21 (or swept) EMA crossover, AbstractTickerStrategy,
CrossDetector-based, long-only. binance / BTC/USDT in all cases.

1. CAGR overflows to a nonsensical value

Single backtest, 2026-08-10 → 2026-08-12:

P&L:          +25.6567
Trades:       1819 (win rate: 0.5%)
Sharpe:       0.446 | Sortino: 4.022
CAGR:         14150890090306516000000.00%
Max Drawdown: 0.08%

Every other metric on this run looks plausible. CAGR alone is off by ~20 orders of magnitude —
looks like an annualization exponent blowing up on a very short window (2 days), e.g.
(1+return)^(365/days) with days near zero or a unit mismatch (days vs. some smaller time unit)
feeding the exponent.

2. Max Drawdown reported above 100% on a spot (unleveraged) instrument

Two sweeps over the same window (2026-06-01 → 2026-06-05), grid and random samplers, varying
ema.fast/ema.slow — every single row in both leaderboards has maxDD% > 100%:

#1 params={ema.fast=15, ema.slow=30}   maxDD%=619.7165   pnl=-61971.6490   trades=3816
#2 params={ema.fast=15, ema.slow=20}   maxDD%=743.0957   pnl=-74309.5675   trades=4514
#3 params={ema.fast=10, ema.slow=30}   maxDD%=736.9358   pnl=-73693.5837   trades=4344
#4 params={ema.fast=5,  ema.slow=20}   maxDD%=1084.6523  pnl=-108465.2274  trades=6027
...

(full leaderboards available on request — happy to attach). BTC/USDT on binance is spot, no
margin/leverage in play, so equity going more than 100% underwater from peak shouldn't be
representable at all with a long-only strategy and no leverage — implies negative account equity.
Consistent double-digit-to-quadruple-digit maxDD% across every row of two independent sweeps
(12 total parameter combinations) suggests a systematic drawdown calculation issue rather than
one bad run.

3. Win rate and P&L directionally inconsistent on the same run

Same run as #1: 1819 trades, win rate 0.5%, but P&L is positive (+25.6567) and Sharpe is
positive (0.446). With a win rate that low, positive P&L requires winning trades to be roughly
200x the size of losing trades on average — possible in principle (huge asymmetric payoff) but
worth double-checking against the trade log, because it's exactly the kind of number a rounding or
field-mapping bug (e.g. win rate computed against the wrong trade set, or as wins / windowTicks
instead of wins / trades) would produce.

Ask

Would help to know whether these three share one root cause (e.g. a shared metrics-aggregation
step run after the backtest itself) or are independent — happy to pull more sample runs/full
leaderboards if useful for repro.

[leonardojgv]

Additional evidence: sweep metrics diverge from submit_backtest metrics for the identical underlying run

Ran into the same signature described above (large PnL, maxDD% > 100%) while diagnosing a
separate parameter-injection bug in submit_sweep (filed as #7). That bug turned
out to make this comparison unusually clean: all 11 rows of an 11-point grid sweep executed with
the same effective parameter value (the sweep never actually applied the requested per-run
values — see that issue for the full repro), so the sweep's own leaderboard row and a plain
submit_backtest of literally the same strategy/params/window can be compared trade-for-trade.

Setup: binance / BTC/USDT, 2026-08-11 → 2026-08-13, identical strategy source in both
calls:

	trades	PnL	Sharpe	MaxDD%	win rate
submit_backtest (standalone)	4,454	-0.8717	-0.008	1.88%	0.5%
submit_sweep row (same effective run)	4,454	-89,122.4561	-8.1361	891.2246%	0.0%

Identical trade count → same underlying simulated run. The standalone numbers look sane (PnL
reads as a percent-of-equity move, Sharpe in a plausible range, maxDD well under 100% on an
unleveraged spot instrument). The sweep's numbers for that same run look like they're computed
on a completely different scale — consistent with your maxDD% > 100% and PnL-in-the-tens-of-
thousands observations in item 2 above. Win rate also flips from 0.5% to 0.0% between the two
reports for the same trades, which may be related to item 3 (win-rate/PnL inconsistency) — worth
checking whether win rate is being computed off a different trade set than PnL/maxDD in the
sweep path.

This narrows things down a bit: whatever's causing the scale/consistency problems seems specific
to the sweep result-aggregation path, since the plain submit_backtest path reports plausible
numbers for the exact same simulated trades.

Environment: qtsurfer-mcp 0.10.3, API https://api.qtsurfer.net/v1

[mrmx]

Independent corroboration — I reproduced the exact same signature on qtsurfer-mcp 0.10.3/0.10.4:

A plain EMA(9)×EMA(21) long-only backtest on binance / BTC/USDT (short window, a few days) returned:

P&L: +338.66 | Trades: 38,564 | win rate 0.4%
Sharpe: 0.157 | MaxDD: 1.35% | CAGR: 1223878987241.82%

Same shape as item 1 here: CAGR off by ~20 orders of magnitude on a short window while the rest of the metrics look plausible. Also confirms the win-rate/PnL consistency concern — a win rate in the single-digit percentage range with a positive P&L and near-zero MaxDD is internally inconsistent, points to win rate being computed off a different trade set than PnL.

The trade-for-trade comparison you added (standalone backtest vs. sweep row on the identical run: same 4,454 trades, PnL -0.8717 / maxDD 1.88% vs PnL -89,122 / maxDD 891%) is the strongest evidence this is a sweep-path result-aggregation bug, not an engine simulation bug. Still open — worth prioritizing since it makes every sweep leaderboard unreadable.

[mrmx]

Retest 2026-08-20 (0.10.4) — the metric bug is still present:

Plain submit_backtest, EMA(9)/EMA(21), BTC/USDT 2026-06-01→06-10:

P&L +123.09 | Trades 13,675 | win 0.4% | Sharpe 0.133 | MaxDD 1.35%
CAGR: 1396261857275970600%

CAGR still overflows on a short window. Every sweep row also reports maxDD% in the hundreds-to-thousands range (1119%, 2058%) on unleveraged spot with win% ~0.02 — internally inconsistent with the plain-backtest path for the same trades.

This and #7 share the same root symptom: sweep-path metrics are computed on a different scale than the standalone backtest path. Worth fixing together.

[mrmx]

Retest 2026-08-21 after engine update — still broken, now over-corrected. The CAGR/maxDD numeric overflow is gone (clamped), BUT every strategy on every window now returns P&L -10000 | MaxDD 100% | CAGR -10000% | win 0% — total account loss regardless of the strategy. EMA (previously P&L +123, Sharpe 0.133) and RSI both report identical full-blowup numbers. The clamp now forces -100% instead of overflowing. Position sizing / P&L normalization still needs a real fix, not a clamp.