Fix mojibake when double byte text is written under a single byte font encoding

[EugineD]

Problem

When an encapsulated-HTML RTF body declares a double byte document code page (\ansicpg932, \ansicpg949, ...) but part of the text is written while a \fcharset0 font is active, that text is silently turned into mojibake. There is no ? and no U+FFFD, because a single byte code page maps nearly every byte, so the corruption is easy to miss.

This is #529. Using the test1.msg attached there, the body text あああ (U+3042 x3, Shift-JIS bytes 82A0 82A0 82A0) comes out of BodyHtml as U+201A U+00A0 x3 — which is exactly those same bytes decoded as cp1252.

The reporter noticed it happens after a NO-BREAK SPACE, and that turns out to be mechanically correct. Word emits the   as {\f3\'a0}, and \f3 is the only \fcharset0 font involved; the Japanese text on the next line carries no font of its own and inherits that single byte encoding.

Cause

TryDecode is only ever called from one place:

if (FontTable.MixedEncodings && _runtimeEncoding.IsSingleByte && byteBuffer.Count > 1 && byteBuffer[0] >= 0x80)
    stringBuilder.Append(TryDecode(byteBuffer));

So by construction, whenever it runs, a high byte has arrived while a single byte font encoding is active. The runtime encoding is already suspect — that is the whole reason TryDecode exists — so falling back to it cannot recover the text.

Charset detection does not help here either: the byte buffer is flushed per high byte run, so CharsetDetector typically sees only about two bytes and detects nothing. Execution drops to the fallback. (Lowering CharsetDetectionEncodingConfidenceLevel therefore has no effect — I tested 0.90 down to 0.01 and the output was byte-identical.)

Fix

Prefer the declared document code page in the one case where it is provably better — the runtime encoding is single byte and cannot represent double byte characters, while the document code page is multi byte and can:

if (!_documentCodePageDeclared || _defaultEncoding == null ||
    _defaultEncoding.IsSingleByte || !RuntimeEncoding.IsSingleByte)
    return RuntimeEncoding;

return _defaultEncoding;

This is intentionally narrow, so existing behaviour is preserved everywhere else:

• both encodings single byte (e.g. \ansicpg1252 with a \fcharset204 font) — unchanged;
• runtime encoding multi byte — unchanged;
• no \ansicpg declared — unchanged, via the new _documentCodePageDeclared flag. This guard matters because _defaultEncoding is seeded with Encoding.Default, which is UTF-8 (multi byte) on .NET Core, and would otherwise start winning for documents that never declared a code page.

Successful detection paths are untouched — a confident detection found in the font table still wins.

Tests

Three tests added to RtfDocumentTests, all synthetic inline RTF, no binary fixtures:

Test	Purpose
MixedCharsetFontsWithNoBreakSpaceBeforeDoubleByteText	Reproduces #529 — \ansicpg932 with the  -induced {\f1\'a0} font switch
MixedCharsetFontsWithDoubleByteDocumentCodePage	The same defect with \ansicpg949 (Korean)
MixedCharsetFontsWithSingleByteDocumentCodePageKeepsFontEncoding	Guard — a Cyrillic \fcharset204 font under \ansicpg1252 must keep the font encoding

Both new failing tests genuinely fail on unpatched master (Expected string length 3 but was 6 and 5 but was 10 — each double byte character becoming two single byte ones), and the guard test passes both before and after, confirming the change is narrow.

Full suite passes on net462, net8.0, net9.0 and net10.0: 59 passed, 0 failed, 2 skipped (up from 56). MsgReader.csproj builds clean in Release across all six target frameworks.

Side observation (not fixed here)

{\f3\'a0} is a scoped group, so the font should revert at the }. It does not: GroupStart and GroupEnd are no-ops in the main loop, and _runtimeEncoding is never saved or restored with group nesting, so the font's encoding leaks past the group. That is a separate and older issue, which was harmless while the fallback was the document code page. I have left it alone to keep this PR focused, but it may be worth a look.

[EugineD]

Friendly ping on this one — no rush at all if you're busy, just flagging it in case it slipped by.

A short summary for whenever you get to it:

• Fixes Japanese multi-byte characters are garbled when appearing on the line following a NO-BREAK SPACE (U+00A0) #529 (Japanese text garbled on the line after a non-breaking space). I confirmed at byte level that the corruption in that report and this change line up exactly — the mangled run re-encodes to precisely the original cp932 bytes.
• The change is deliberately narrow. It only alters the fallback inside TryDecode, which is reached solely when the font table has mixed encodings and a high byte arrives while a single-byte font encoding is active — i.e. only when that runtime encoding is already known to be unable to represent the text. Every other path keeps the current behaviour, so it is not a blunt revert of 895d851.
• Three regression tests added, all synthetic inline RTF, no binary fixtures. Locally: 59 passed / 0 failed / 2 skipped on net462, net8.0, net9.0 and net10.0. The two new failing-before tests were verified to genuinely fail without the fix.

Two notes on the CI, since the checks look worse than they are:

• The Test .NET Libraries workflow is still waiting on maintainer approval (first-time contributor), so it hasn't actually run yet.
• The AppVeyor check is red, but it appears to be failing repo-wide rather than because of this PR — the same failure shows up on recent master commits with no PR attached (MSB4041 on MsgViewer.csproj; the pinned MSBuild can't parse SDK-style projects). Happy to look at that separately if it would be useful.

Also mentioned in the description: GroupStart / GroupEnd are currently no-ops, so _runtimeEncoding isn't scoped to RTF groups and can leak past a closing brace. I left that untouched to keep this PR focused, but I'm glad to follow up on it separately if you'd like it addressed.