Fix unhandled InvalidCastException when a rewrite/metathesis rule has a top-level Quantifier#464
Open
johnml1135 wants to merge 1 commit into
Open
Fix unhandled InvalidCastException when a rewrite/metathesis rule has a top-level Quantifier#464johnml1135 wants to merge 1 commit into
johnml1135 wants to merge 1 commit into
Conversation
…el Quantifier Constraint<TData, TOffset> and Quantifier<TData, TOffset> are both direct subclasses of PatternNode<TData, TOffset> (src/SIL.Machine/Matching/Constraint.cs:12-13, src/SIL.Machine/Matching/Quantifier.cs:13-14) -- siblings, not related by inheritance. The DTD's PhoneticSequence element allows an OptionalSegmentSequence (loaded by XmlLanguageLoader as a Quantifier) in exactly the same positions as a plain Segment/SimpleContext (HermitCrabInput.dtd:515, 560), including as the target (Lhs) or replacement (Rhs) of a rewrite rule, or as a metathesis switch group -- not just in a rule's environment. The rule specs assumed every top-level Lhs/Rhs child was a Constraint and cast accordingly with `Children.Cast<Constraint<Word, ShapeNode>>()` or a direct cast, so a DTD-legal Quantifier there crashed with an unhandled InvalidCastException: - PhonologicalRules/SynthesisRewriteRuleSpec.cs:33 (and :93, lazily) - PhonologicalRules/FeatureAnalysisRewriteRuleSpec.cs:42-43 - PhonologicalRules/NarrowAnalysisRewriteRuleSpec.cs:45 (lazily, in Unapply) - PhonologicalRules/EpenthesisAnalysisRewriteRuleSpec.cs:18 - PhonologicalRules/SynthesisMetathesisRuleSpec.cs:31 - PhonologicalRules/AnalysisMetathesisRuleSpec.cs:53 Depending on which of these ran first, the raw InvalidCastException surfaced either unhandled directly out of `new Morpher(...)` (the synthesis side has no try/catch around rule compilation), or already wrapped in a CompileException by the pre-existing catch-all in AnalysisStratumRule.CompilePhonologicalRule -- but even then the message was uninformative (just named the rule; the actual cause was buried in an opaque InnerException). Fix: add PatternNodeCastExtensions (CastToConstraints/AsConstraint) and use it at every one of the sites above instead of the unchecked cast. Behavior for any grammar built entirely from Constraints (i.e. every currently-working grammar) is unchanged -- the helper returns the same Constraint it would have cast to. When a Quantifier (or any other non-Constraint PatternNode) shows up, it now raises a clear, typed CompileException naming the unsupported construct and where it appeared (e.g. "The target (left-hand side) of a rewrite rule contains a 'Quantifier`2', which is not supported there...") instead of an opaque InvalidCastException. Genuinely supporting a Quantifier in this position was considered but rejected as too large a semantic change (matching/deletion of a variable-width sequence during rewrite application) for this fix; a clear diagnostic is the conservative, reviewable option. Adds two regression tests to RewriteRuleTests.cs (QuantifierInTargetIsRejectedWithClearError, QuantifierInReplacementIsRejectedWithClearError) that build such a rule directly (bypassing the XML loader, matching the rest of the file) and confirm constructing a Morpher over it now fails fast with the new CompileException. Verified both fail (one with a raw InvalidCastException, one via the pre-existing CompileException wrapper around it) against the unmodified code, and pass after the fix. Found while building an independent FST-based morphological analyzer/proposer (PanGloss) against this library -- a real grammar under construction hit this, not a hypothetical. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #464 +/- ##
==========================================
+ Coverage 73.29% 73.33% +0.03%
==========================================
Files 444 445 +1
Lines 37283 37313 +30
Branches 5116 5118 +2
==========================================
+ Hits 27328 27363 +35
+ Misses 8830 8825 -5
Partials 1125 1125 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Constraint<TData, TOffset>(src/SIL.Machine/Matching/Constraint.cs:12-13) andQuantifier<TData, TOffset>(src/SIL.Machine/Matching/Quantifier.cs:13-14) are both direct subclasses ofPatternNode<TData, TOffset>— siblings, with no inheritance relationship between them.The HermitCrab rule specs assume every top-level child of a rule's target (Lhs) or replacement (Rhs) pattern — or a metathesis switch group — is a
Constraint, and cast accordingly withChildren.Cast<Constraint<Word, ShapeNode>>()or a direct(Constraint<Word, ShapeNode>)cast:PhonologicalRules/SynthesisRewriteRuleSpec.cs:33(and:93, lazily, viaPattern.Acceptable)PhonologicalRules/FeatureAnalysisRewriteRuleSpec.cs:42-43PhonologicalRules/NarrowAnalysisRewriteRuleSpec.cs:45(lazily, inUnapply)PhonologicalRules/EpenthesisAnalysisRewriteRuleSpec.cs:18PhonologicalRules/SynthesisMetathesisRuleSpec.cs:31PhonologicalRules/AnalysisMetathesisRuleSpec.cs:53But the DTD's
PhoneticSequenceelement (HermitCrabInput.dtd:515) allows anOptionalSegmentSequencein exactly the same positions as a plainSegment/SimpleContext— andOptionalSegmentSequenceitself (HermitCrabInput.dtd:560) carries anidattribute explicitly documented as "only needed for a metathesis rule", so it's legal there too.XmlLanguageLoader.LoadPatternNodes(~1405-1415, ~1493-1505) builds a realQuantifierfor anOptionalSegmentSequencewith no LHS-vs-environment distinction.So a DTD-legal grammar — a rewrite rule whose target or replacement contains a top-level optional/repeated segment sequence — crashes with an unhandled
InvalidCastExceptionduringMorpherconstruction. Depending on which of the sites above runs first, it either propagates raw straight out ofnew Morpher(...)(the synthesis side has no try/catch around rule compilation), or gets caught by a pre-existing, unrelated catch-all inAnalysisStratumRule.CompilePhonologicalRule— but even then the message is uninformative (just names the rule; the real cause is buried in an opaqueInnerException).Quantifiers already work correctly in a rule's environment (
LeftEnvironment/RightEnvironment— see the existingQuantifierRulestest) since environment matching doesn't go through this cast. The bug is specific to the target/replacement/switch-group position.Reproduction
Verified directly (not hypothetical) — found while building an independent FST-based morphological analyzer/proposer (PanGloss) against this library; a grammar under construction hit this.
Two new tests in
RewriteRuleTests.csbuild aRewriteRuledirectly (bypassing the XML loader, matching the rest of that file) with a top-levelQuantifierin the Lhs and, separately, the Rhs, then construct aMorpher:QuantifierInTargetIsRejectedWithClearError— before the fix, throws a raw, unhandledInvalidCastExceptionstraight out ofnew Morpher(...)(confirmed against unmodifiedmaster).QuantifierInReplacementIsRejectedWithClearError— before the fix, throws aCompileException(via the pre-existingAnalysisStratumRulewrapper) whoseInnerExceptionis the same raw, uninformativeInvalidCastException(also confirmed against unmodifiedmaster).The fix
I considered two approaches:
Quantifierin this position — the DTD permits it, so the library arguably should handle it.I went with (2). Actually supporting a variable-width quantifier as the target of a rewrite rule (matched-and-replaced, or matched-and-deleted) is a real semantic feature, not a mechanical fix — it changes what "the same number of Lhs and Rhs children" even means for the whole rewrite-subrule-shape dispatch (
Feature/Narrow/Epenthesisselection) these classes are built around. That's too large and semantically risky a change to make confidently in an external repo I don't own, especially with no existing test/grammar exercising it to pin the intended behavior. A clear, fail-fast diagnostic is the conservative, reviewable option, and is a strict improvement over an opaque crash either way.Added
PatternNodeCastExtensions(CastToConstraints/AsConstraint) and used it at every site above instead of the unchecked cast. For any grammar built entirely fromConstraints — i.e. every currently-working grammar — behavior is byte-for-byte unchanged: the helper returns the exact sameConstraintthe cast would have produced. When aQuantifier(or any other non-ConstraintPatternNode) shows up in one of these positions, it now raises aCompileExceptionsuch as:Test evidence
Full solution, before vs. after (temporarily reverted just the 6 fixed
src/files tomaster's content to confirm red state, then restored):Before the fix (unmodified
master+ the two new tests): both new tests fail —QuantifierInTargetIsRejectedWithClearError: rawSystem.InvalidCastExceptionthrown directly fromnew Morpher(...), atSynthesisRewriteRuleSpec.cs:33.QuantifierInReplacementIsRejectedWithClearError:CompileExceptionwrapping a rawInvalidCastExceptionatFeatureAnalysisRewriteRuleSpec.cs:43.After the fix: full solution build clean (0 errors, 0 new warnings),
dotnet teston the whole solution:All pre-existing tests pass unchanged; only the two new tests are added.
Ran
dotnet csharpier formaton the touched files only (no wholesale reformatting).Test plan
masterdotnet build: cleandotnet test: all pass (962 total, 3 pre-existing unrelated skips)dotnet csharpier checkclean on touched files🤖 Generated with Claude Code
This change is