Why
generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt (1394 lines) derives "how do
I resolve a Var in this query block" three times, once per entry point, with the same six-step
fallback chain hand-copied into three lambdas. Three comments literally say "See analyzeNodeTree's
identical guard". Any future rule (a new RTE kind, a new narrowing exception) has to be added in three
places and can drift. Collapsing the three into one value object deletes roughly 150 lines, removes
two helper functions that exist only because the lambdas were duplicated, and brings the file under
1000 lines.
Current state
The three sites and what each computes:
| Site |
Lines |
Derives |
isSourceColumnNotNull chain |
analyzeNodeTree |
440-597 |
rangeTable, hasGroupingSets, groupRteMap, resolvedCtes, subqueryColumnNotNull, cteColumnNotNull (via buildCteColumnNotNull, 894-910), qualNotNullVars gated on applyQualNarrowing && !hasGroupingSets && resultRelationVarno == 0, forceNewNullable |
merge-absent → quals → base relid → group-RTE remap → subquery ∥ cte |
buildCteBodyAnalyzer |
1102-1171 |
same, cte map via buildInnerCteNotNull (1072-1087), qual gate uses !isDml |
identical chain |
analyzeQueryBlockNullability |
1257-1316 |
same, cte lookup inlined at 1305-1311, no merge check, qual gate has no DML check, no forceNewNullable |
identical chain minus merge step |
Differences are all incidental, not semantic:
- CTE lookup.
buildCteColumnNotNull ignores ctelevelsup and reads resolvedCtes. The
outermost statement's RTEs are always ctelevelsup 0 (KDoc at 888-892), so "if ctelevelsup == 0
read own CTEs else read enclosing CTEs, with enclosing = empty" gives the same answer there.
buildInnerCteNotNull and the inline block at 1305-1311 already are that rule.
- Merge check.
analyzeQueryBlockNullability handles a SubLink subselect or a FROM subquery.
Neither can be a MERGE, so an empty mergeAbsentVarnos is equivalent.
- DML qual gate. Same reasoning: a subselect is always a
SELECT, so
parseResultRelation(queryBlock) == 0 and the gate is a no-op there.
forceNewNullable. forcesNewNullable(SELECT block) is false, matching the current default.
Target state
One private class and one builder replace the three lambdas and two map-builders:
/** Everything needed to answer isSourceColumnNotNull for one query block. */
private class QueryBlockScope(
val rangeTable: Map<Int, Int>,
val hasGroupingSets: Boolean,
val groupRteMap: Map<Pair<Int, Int>, Pair<Int, Int>>,
val qualProvenVars: Set<Pair<Int, Int>>,
val ownCtes: Map<String, List<Boolean>>,
val enclosingCtes: Map<String, List<Boolean>>,
val cteReferences: Map<Int, NodeTreeCteReference>,
val subqueryColumnNotNull: Map<Pair<Int, Int>, Boolean>,
val mergeAbsentVarnos: Map<Int, Boolean>,
val forceNewNullable: Boolean,
) {
fun isSourceColumnNotNull(varno: Int, varattno: Int, isColumnNotNull: (Pair<Int, Int>) -> Boolean): Boolean {
if (mergeAbsentVarnos[varno] == true) return false
if (isProvenByQuals(qualProvenVars, groupRteMap, varno, varattno)) return true
rangeTable[varno]?.let { relid -> return isColumnNotNull(relid to varattno) }
groupRteMap[varno to varattno]?.let { (baseVarno, baseAttno) ->
val baseRelid = rangeTable[baseVarno] ?: return false
return isColumnNotNull(baseRelid to baseAttno)
}
if (subqueryColumnNotNull[varno to varattno] == true) return true
val reference = cteReferences[varno] ?: return false
val ctesInScope = if (reference.ctelevelsup == 0) ownCtes else enclosingCtes
return ctesInScope[reference.name]?.getOrNull(varattno - 1) == false
}
}
private fun buildQueryBlockScope(
queryBlock: String,
enclosingCtes: Map<String, List<Boolean>>,
applyQualNarrowing: Boolean,
sql: String,
depth: Int = SUBLINK_ANALYSIS_DEPTH_BUDGET,
mergeAbsentVarnos: Map<Int, Boolean> = emptyMap(),
): QueryBlockScope
buildQueryBlockScope does, in order: parseRangeTable, hasGroupingSets, groupRteMap (empty
when grouping sets), resolveCteBodies(queryBlock, applyQualNarrowing, sql) for ownCtes,
buildSubqueryColumnNotNull(queryBlock, ownCtes, applyQualNarrowing, sql, depth),
parseCteRangeTableEntries, qualProvenNonNullVars gated on
applyQualNarrowing && !hasGroupingSets && parseResultRelation(queryBlock) == 0, and
forcesNewNullable(queryBlock).
buildAnalyzer gains an overload buildAnalyzer(scope: QueryBlockScope, depth: Int) that wires
scope::isSourceColumnNotNull (partially applied with ::isColumnNotNull), scope.hasGroupingSets,
scope.forceNewNullable, applyQualNarrowing, depth, and scope.ownCtes as resolvedCtes.
Then:
analyzeNodeTree builds one scope (enclosingCtes = emptyMap(), mergeAbsentVarnos from its
parameter) and keeps only what is unique to it: the targetListByResno substitution and the
RETURNING-first read. Its plainIsSourceColumnNotNull becomes
{ varno, varattno -> scope.isSourceColumnNotNull(varno, varattno, ::isColumnNotNull) }.
buildCteBodyAnalyzer becomes buildAnalyzer(buildQueryBlockScope(queryBlock, previouslyResolved, applyQualNarrowing, sql, mergeAbsentVarnos = mergeAbsentVarnos), depth = default).
analyzeQueryBlockNullability becomes buildAnalyzer(buildQueryBlockScope(queryBlock, resolvedCtes, applyQualNarrowing, sql, depth), depth).extractColumnNullability(queryBlock).
- Delete
buildCteColumnNotNull, buildInnerCteNotNull, and every "See analyzeNodeTree's identical
guard" comment. The reasoning those comments carried (grouping-set null extension defeats qual
narrowing; a DML body's WHERE can test a column its SET overwrites) moves once onto
buildQueryBlockScope's KDoc.
isProvenByQuals stays as-is (it is already shared).
Test design
Behavior must not change. Pins:
QueryAnalysisTest (545 live-PostgreSQL tests) and the golden scenarios are the primary pin.
- Before refactoring, confirm the three "incidental difference" claims above each have a covering
test, and add one where missing, so the unification is proven rather than argued:
- CTE referenced from inside a
FROM subquery (ctelevelsup 1 in the subquery's own rtable):
WITH c AS (SELECT id FROM t) SELECT s.id FROM (SELECT id FROM c) s with t.id NOT NULL →
id NOT NULL. Grep QueryAnalysisTest.kt for ctelevelsup; add under
CommonTableExpressions if absent.
- A nested
WITH shadowing an enclosing CTE of the same name inside a CTE body
(buildInnerCteNotNull's reason for existing). Add if absent.
- A
SubLink subselect whose body itself has a WHERE x IS NOT NULL narrowing (proves the qual
gate still applies inside analyzeQueryBlockNullability). Add if absent.
- No new test may assert on private structure; all pins go through
JdbcAnalyzer.analyzeQuery as the
existing tests do.
Acceptance criteria
ColumnNullabilityAnalyzer.kt contains exactly one isSourceColumnNotNull implementation and no
buildCteColumnNotNull / buildInnerCteNotNull.
- The three tests above exist (pre-existing or added) and pass.
./gradlew :generator:check :gradle-plugin:test passes.
- Golden regeneration leaves
test-scenarios* unchanged.
ColumnNullabilityAnalyzer.kt is under 1000 lines.
Files
generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt
generator/src/test/kotlin/norm/generator/QueryAnalysisTest.kt (additions only)
Conventions every issue inherits
- Repo root:
/Volumes/Code/3rd-party/norm. Module under change is almost always generator/.
- Style: 2-space indent, 120 columns, full words in identifiers (
parameter, not param), no
section-separator comments, @Nested classes group tests. Format with ./gradlew spotlessApply.
- TDD: tests are written first or alongside. For a behavior-preserving refactor the existing suite is
the pin; each issue says which tests must also be added.
- Golden files under
test-scenarios*/ are never hand-edited. A refactor is behavior-preserving only
if ./gradlew :gradle-plugin:generateGoldenFiles leaves git status --porcelain test-scenarios test-scenarios-frameworks empty.
- Verification for a
generator/ change: ./gradlew :generator:check :gradle-plugin:test (Docker
required). Then the golden regeneration check above.
- Commit message explains the design decision, not the diff.
- Never run
./gradlew clean or disable the configuration/build cache to "fix" a build problem.
Why
generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt(1394 lines) derives "how doI resolve a
Varin this query block" three times, once per entry point, with the same six-stepfallback chain hand-copied into three lambdas. Three comments literally say "See analyzeNodeTree's
identical guard". Any future rule (a new RTE kind, a new narrowing exception) has to be added in three
places and can drift. Collapsing the three into one value object deletes roughly 150 lines, removes
two helper functions that exist only because the lambdas were duplicated, and brings the file under
1000 lines.
Current state
The three sites and what each computes:
isSourceColumnNotNullchainanalyzeNodeTreerangeTable,hasGroupingSets,groupRteMap,resolvedCtes,subqueryColumnNotNull,cteColumnNotNull(viabuildCteColumnNotNull, 894-910),qualNotNullVarsgated onapplyQualNarrowing && !hasGroupingSets && resultRelationVarno == 0,forceNewNullablebuildCteBodyAnalyzerbuildInnerCteNotNull(1072-1087), qual gate uses!isDmlanalyzeQueryBlockNullabilityforceNewNullableDifferences are all incidental, not semantic:
buildCteColumnNotNullignoresctelevelsupand readsresolvedCtes. Theoutermost statement's RTEs are always
ctelevelsup 0(KDoc at 888-892), so "ifctelevelsup == 0read own CTEs else read enclosing CTEs, with enclosing = empty" gives the same answer there.
buildInnerCteNotNulland the inline block at 1305-1311 already are that rule.analyzeQueryBlockNullabilityhandles aSubLinksubselect or aFROMsubquery.Neither can be a
MERGE, so an emptymergeAbsentVarnosis equivalent.SELECT, soparseResultRelation(queryBlock) == 0and the gate is a no-op there.forceNewNullable.forcesNewNullable(SELECT block)isfalse, matching the current default.Target state
One private class and one builder replace the three lambdas and two map-builders:
buildQueryBlockScopedoes, in order:parseRangeTable,hasGroupingSets,groupRteMap(emptywhen grouping sets),
resolveCteBodies(queryBlock, applyQualNarrowing, sql)forownCtes,buildSubqueryColumnNotNull(queryBlock, ownCtes, applyQualNarrowing, sql, depth),parseCteRangeTableEntries,qualProvenNonNullVarsgated onapplyQualNarrowing && !hasGroupingSets && parseResultRelation(queryBlock) == 0, andforcesNewNullable(queryBlock).buildAnalyzergains an overloadbuildAnalyzer(scope: QueryBlockScope, depth: Int)that wiresscope::isSourceColumnNotNull(partially applied with::isColumnNotNull),scope.hasGroupingSets,scope.forceNewNullable,applyQualNarrowing,depth, andscope.ownCtesasresolvedCtes.Then:
analyzeNodeTreebuilds one scope (enclosingCtes = emptyMap(),mergeAbsentVarnosfrom itsparameter) and keeps only what is unique to it: the
targetListByResnosubstitution and theRETURNING-first read. ItsplainIsSourceColumnNotNullbecomes{ varno, varattno -> scope.isSourceColumnNotNull(varno, varattno, ::isColumnNotNull) }.buildCteBodyAnalyzerbecomesbuildAnalyzer(buildQueryBlockScope(queryBlock, previouslyResolved, applyQualNarrowing, sql, mergeAbsentVarnos = mergeAbsentVarnos), depth = default).analyzeQueryBlockNullabilitybecomesbuildAnalyzer(buildQueryBlockScope(queryBlock, resolvedCtes, applyQualNarrowing, sql, depth), depth).extractColumnNullability(queryBlock).buildCteColumnNotNull,buildInnerCteNotNull, and every "See analyzeNodeTree's identicalguard" comment. The reasoning those comments carried (grouping-set null extension defeats qual
narrowing; a DML body's WHERE can test a column its SET overwrites) moves once onto
buildQueryBlockScope's KDoc.isProvenByQualsstays as-is (it is already shared).Test design
Behavior must not change. Pins:
QueryAnalysisTest(545 live-PostgreSQL tests) and the golden scenarios are the primary pin.test, and add one where missing, so the unification is proven rather than argued:
FROMsubquery (ctelevelsup 1in the subquery's own rtable):WITH c AS (SELECT id FROM t) SELECT s.id FROM (SELECT id FROM c) switht.id NOT NULL→idNOT NULL. GrepQueryAnalysisTest.ktforctelevelsup; add underCommonTableExpressionsif absent.WITHshadowing an enclosing CTE of the same name inside a CTE body(
buildInnerCteNotNull's reason for existing). Add if absent.SubLinksubselect whose body itself has aWHERE x IS NOT NULLnarrowing (proves the qualgate still applies inside
analyzeQueryBlockNullability). Add if absent.JdbcAnalyzer.analyzeQueryas theexisting tests do.
Acceptance criteria
ColumnNullabilityAnalyzer.ktcontains exactly oneisSourceColumnNotNullimplementation and nobuildCteColumnNotNull/buildInnerCteNotNull../gradlew :generator:check :gradle-plugin:testpasses.test-scenarios*unchanged.ColumnNullabilityAnalyzer.ktis under 1000 lines.Files
generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.ktgenerator/src/test/kotlin/norm/generator/QueryAnalysisTest.kt(additions only)Conventions every issue inherits
/Volumes/Code/3rd-party/norm. Module under change is almost alwaysgenerator/.parameter, notparam), nosection-separator comments,
@Nestedclasses group tests. Format with./gradlew spotlessApply.the pin; each issue says which tests must also be added.
test-scenarios*/are never hand-edited. A refactor is behavior-preserving onlyif
./gradlew :gradle-plugin:generateGoldenFilesleavesgit status --porcelain test-scenarios test-scenarios-frameworksempty.generator/change:./gradlew :generator:check :gradle-plugin:test(Dockerrequired). Then the golden regeneration check above.
./gradlew cleanor disable the configuration/build cache to "fix" a build problem.