fix: validate DH GEX request bounds

[MarkAtwood]

Summary

DoKexDhGexRequest() in src/internal.c reads three client-controlled values from the SSH_MSG_KEX_DH_GEX_REQUEST message — dhGexMinSz, dhGexPreferredSz, and dhGexMaxSz (via three GetUint32 calls) — and then proceeds to SendKexDhGexGroup() without validating them. There is no check that the values satisfy min <= preferred <= max and no bounds check against the library's supported DH group sizes.

Per RFC 4419, these requested sizes are folded into the key-exchange hash H (dhGexMinSz is used when computing the exchange hash). Accepting nonsensical or out-of-range values invites protocol confusion and divergence of H between peers.

Fix

Add a validation block after the three sizes are parsed and before proceeding. It rejects the request (aborts the handshake with WS_BAD_ARGUMENT) when:

• dhGexMinSz < WOLFSSH_DEFAULT_GEXDH_MIN (1024)
• dhGexMaxSz > WOLFSSH_DEFAULT_GEXDH_MAX (8192)
• dhGexMinSz > dhGexPreferredSz
• dhGexPreferredSz > dhGexMaxSz

The bound macros are the project's own existing defaults from wolfssh/internal.h. The request is rejected rather than clamped on purpose: because the requested values feed the RFC 4419 exchange hash H, silently clamping would make our H diverge from the client's and break the handshake in a harder-to-diagnose way.

Verification

Built with --enable-all (make, exit 0) against a local wolfSSL install; the modified translation unit compiles cleanly. The affected path is live under WOLFSSH_NO_DH_GEX_SHA256 not being defined.

Reported by static analysis (Fenrir finding 60).

[Copilot]

Pull request overview

This PR hardens the DH Group Exchange (GEX) key-exchange path by validating client-provided size parameters in DoKexDhGexRequest() before proceeding to select and send a DH group, preventing nonsensical/out-of-policy requests from influencing the RFC 4419 exchange hash inputs.

Changes:

• Add bounds and ordering validation for dhGexMinSz, dhGexPreferredSz, and dhGexMaxSz against WOLFSSH_DEFAULT_GEXDH_MIN/MAX.
• Reject invalid DH GEX requests early with WS_BAD_ARGUMENT (rather than clamping) to avoid hash divergence between peers.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[aidangarske]

🐺 Skoll Code Review

Overall recommendation: REQUEST_CHANGES
Findings: 2 total — 2 posted, 0 skipped

Posted findings

• [High] DH GEX validation accepts ranges the server cannot satisfy — src/internal.c:7256-7263
• [Medium] New DH GEX request validation has no regression coverage — src/internal.c:7256-7263

Review generated by Skoll.

[aidangarske]

🟠 [High] DH GEX validation accepts ranges the server cannot satisfy
🚫 BLOCK bug

The new validation accepts any ordered request inside WOLFSSH_DEFAULT_GEXDH_MIN..WOLFSSH_DEFAULT_GEXDH_MAX, then immediately calls SendKexDhGexGroup(). That send path always emits dhPrimeGroup14, a 2048-bit group. A request such as min=3072, preferred=3072, max=4096 passes the new check but cannot be satisfied by the current server implementation, so the server still sends a group outside the client's requested range. The PR-added validation gate should match what this path can actually produce, or select a supported group within the requested range.

Recommendation: Reject unsatisfiable ranges such as dhGexMinSz > 2048 or dhGexMaxSz < 2048 while the GEX server path is hard-coded to group14, or implement group selection before sending.

[aidangarske]

🟡 [Medium] New DH GEX request validation has no regression coverage
💡 SUGGEST test

This PR changes externally visible server behavior by rejecting malformed DH GEX request sizes, but no tests were added. The only existing wolfSSH_TestDoKexDhGexRequest coverage found is the first-packet-follows skip case, which intentionally avoids parsing the payload and does not exercise the new validation block.

Recommendation: Cover the new rejection paths through wolfSSH_TestDoKexDhGexRequest() or an integration-style server handshake test for min < 1024, max > 8192, min > preferred, preferred > max, and at least one accepted boundary case.