fix(personality): relative-frequency cognitive-function scoring + opt-in LLM-vote mode - #19
Merged
Merged
Conversation
… LLM-vote alternative The 8 Jungian cognitive function scores were computed as mean effective-pattern confidence, but confidence has a hard 70 floor by prompt design (see cli/src/analysis/prompts.ts) and clusters 70-95 — so every function with any samples landed in the "high" band almost by construction, producing uniformly high, undifferentiated scores that don't resemble a real MBTI-style profile (an ipsative construct where functions compete, not all score independently high). computeCognitiveFunctions now scores each function by its RELATIVE FREQUENCY SHARE of pattern instances against a "fair share" (1/8) baseline instead — this differentiates by construction whenever category usage isn't perfectly uniform. Bumped PERSONALITY_ANALYSIS_VERSION to 2.1.0 and made readSnapshot invalidate cached rows on analysisVersion mismatch (previously only checked profileVersion), so already-cached mis-scored snapshots get recomputed instead of served stale forever. Also adds an opt-in 'llm-vote' scoring mode (dashboard.analysis.personality. cognitiveFunctionScoring in config.json, toggle via `code-insights config set personality-scoring llm-vote`): calls the LLM N independent rounds (llmVoteRounds, default 3) to score all 8 functions from pattern-count + example evidence and averages the results, as an alternative to the deterministic formula. PersonalityProfile.cognitiveFunctionScoringMode records which mode produced a given profile; MbtiCard surfaces it in the UI. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
ClaudiaFang
marked this pull request as ready for review
July 21, 2026 13:55
ClaudiaFang
pushed a commit
that referenced
this pull request
Jul 23, 2026
Theme docs site to match web app
ClaudiaFang
added a commit
that referenced
this pull request
Jul 23, 2026
…-fix fix(personality): relative-frequency cognitive-function scoring + opt-in LLM-vote mode
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
computeCognitiveFunctionsscored each of the 8 Jungian functions by mean effective-pattern confidence, which has a hard 70 floor by prompt design and clusters 70-95 — so every function with any samples landed in the "high" band almost by construction, regardless of actual relative strength.PERSONALITY_ANALYSIS_VERSIONto 2.1.0 and madereadSnapshotinvalidate cached rows onanalysisVersionmismatch (previously only checkedprofileVersion), so previously-cached mis-scored snapshots recompute instead of serving stale forever.llm-votescoring mode (config:dashboard.analysis.personality.cognitiveFunctionScoring, toggle viacode-insights config set personality-scoring llm-vote/personality-vote-rounds <n>): runs N independent LLM scoring rounds and averages them, as an alternative to the deterministic formula.PersonalityProfile.cognitiveFunctionScoringModerecords which mode produced a profile;MbtiCardshows a small "(LLM-voted)" label when applicable.Test plan
pnpm build— all 3 packages (cli/server/dashboard) build cleanpnpm test— 1599/1599 tests pass (917 cli + 667 server + 15 dashboard), including new coverage for the frequency-share formula, cache invalidation on staleanalysisVersion, andscoreCognitiveFunctionsByLlmVote(averaging, malformed-round handling, zero-evidence forcing null, total-failure fallback)🤖 Generated with Claude Code