Skip to content

fix(embedding): cosineSimilarity returns nil for incomparable spaces, not 0 - #68

Merged
aurascoper merged 1 commit into
feat/dialectic-conformance-fixturefrom
fix/embedding-comparability
Aug 22, 2026
Merged

fix(embedding): cosineSimilarity returns nil for incomparable spaces, not 0#68
aurascoper merged 1 commit into
feat/dialectic-conformance-fixturefrom
fix/embedding-comparability

Conversation

@aurascoper

Copy link
Copy Markdown
Owner

Closes the defect recorded in docs/reviews/embedding-comparability-unenforced-2026-08-12.md: Embedding.cosineSimilarity(to:) guarded on dimension alone and returned 0 for a mismatch, which is also the legitimate score for orthogonal vectors from the same model. A cross-space candidate therefore scored a neutral 0.5 and the loop kept going.

Based on feat/dialectic-conformance-fixture because the review note this resolves lives on that branch, not main.

Change

Mirrors the Rust port (neuralcompose-hypnagogic/src/embedding.rs):

  • Embedding.isComparable(with:): same modelID, same dimension, non-empty. seed/version are deliberately not checked: centroid(of:) builds centroids with seed 0 and must stay comparable to its members.
  • cosineSimilarity(to:) -> Float?, nil when not comparable.
  • DialecticalDynamics.normalizedSimilarity(_:_:) traps on nil naming both spaces; all 12 BCICore scoring sites route through it. One embedder per session makes incomparable a defect, not a state to absorb.
  • centroid(of:) returns nil on any incomparable member instead of averaging the comparable subset silently.
  • Evals force-unwrap (single map, single embedder); tests use XCTUnwrap.
  • New DialecticalDynamicsTests.testIncomparableIsNilNotZero: orthogonal-and-comparable is 0, incomparable is nil, mixed-space centroid is nil. Swift twin of the Rust Some(0.0) != None pin.
  • embedding_contract.md §2.7 now cites the type and the test rather than a doc comment.

Not verified

Written on a Linux host with no Swift toolchain; the package targets macOS 14. swift test has not been run. Please run it on the Mac before merging. If anything fails to compile, the first suspect is XCTAssertEqual(try XCTUnwrap(...), 0) in the new test.

🤖 Generated with Claude Code

https://claude.ai/code/session_01J5MbSKyxJn8jZPoggQjrJT

… not 0

`Embedding.cosineSimilarity(to:)` guarded on dimension alone and returned
`0` for a mismatch, which is also the legitimate score for orthogonal
vectors from the same model. Per docs/reviews/embedding-comparability-
unenforced-2026-08-12.md, that made "incomparable" indistinguishable from
"unrelated" and scored a cross-space candidate a neutral 0.5.

Mirrors the Rust port (neuralcompose-hypnagogic/src/embedding.rs):

- `Embedding.isComparable(with:)`: same modelID, same dimension, non-empty.
  seed/version are not part of it because `centroid(of:)` builds centroids
  with seed 0 and must stay comparable to its members.
- `cosineSimilarity(to:) -> Float?`, nil when not comparable.
- `DialecticalDynamics.normalizedSimilarity(_:_:)` traps on nil with the two
  space identities in the message; all BCICore scoring sites route through
  it. One embedder per session makes incomparable a defect, not a state.
- `centroid(of:)` returns nil on any incomparable member instead of
  silently averaging the comparable subset.
- Evals force-unwrap (single map, single embedder); tests use XCTUnwrap.
- New test `testIncomparableIsNilNotZero`: orthogonal-and-comparable is 0,
  incomparable is nil, mixed-space centroid is nil.
- embedding_contract.md section 2.7 now cites the type and the test, not a
  doc comment.

UNVERIFIED ON THIS HOST: written on Linux with no Swift toolchain; the
package targets macOS 14. Run `swift test` on the Mac before merging.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01J5MbSKyxJn8jZPoggQjrJT
@aurascoper

Copy link
Copy Markdown
Owner Author

Ran on the Mac as requested (macOS 26.5.2, Xcode 26.6, Swift 6.3.3), in a fresh worktree at 1d89103.

swift test --skip MindMonitorOSCStreamTests: exit 0, 441 passed, 8 skipped, 0 failed. The new DialecticalDynamicsTests.testIncomparableIsNilNotZero passed, as did the other PR-touched suites (SemanticReplayRegressionTests, SemanticBGEReplayRegressionTests, SentenceEmbedderTests). The XCTAssertEqual(try XCTUnwrap(...), 0) line compiles and passes. Skips are all environmental (no golden recording, no BGE model, no MLX model dir, NEURALCOMPOSE_MLX_XCODE_BUILT unset, no WorldModelDemo Core ML export).

Unrelated finding, pre-existing: an unfiltered swift test dies silently inside MindMonitorOSCStreamTests.testTruncatedSampleAddressCountsAsDroppedNotIgnored (no failure line, no crash report, exit 1), which aborts every suite sorting after it alphabetically. Reproduces identically on this PR's parent commit 01681b5, so it is not this change; the CI --skip is hiding a deterministic process death, not a port-collision flake. Worth its own issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant