Skip to content

Resolve a query block once in ColumnNullabilityAnalyzer #274

Description

Why

generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt (1394 lines) derives "how do
I resolve a Var in this query block" three times, once per entry point, with the same six-step
fallback chain hand-copied into three lambdas. Three comments literally say "See analyzeNodeTree's
identical guard". Any future rule (a new RTE kind, a new narrowing exception) has to be added in three
places and can drift. Collapsing the three into one value object deletes roughly 150 lines, removes
two helper functions that exist only because the lambdas were duplicated, and brings the file under
1000 lines.

Current state

The three sites and what each computes:

Site Lines Derives isSourceColumnNotNull chain
analyzeNodeTree 440-597 rangeTable, hasGroupingSets, groupRteMap, resolvedCtes, subqueryColumnNotNull, cteColumnNotNull (via buildCteColumnNotNull, 894-910), qualNotNullVars gated on applyQualNarrowing && !hasGroupingSets && resultRelationVarno == 0, forceNewNullable merge-absent → quals → base relid → group-RTE remap → subquery ∥ cte
buildCteBodyAnalyzer 1102-1171 same, cte map via buildInnerCteNotNull (1072-1087), qual gate uses !isDml identical chain
analyzeQueryBlockNullability 1257-1316 same, cte lookup inlined at 1305-1311, no merge check, qual gate has no DML check, no forceNewNullable identical chain minus merge step

Differences are all incidental, not semantic:

  • CTE lookup. buildCteColumnNotNull ignores ctelevelsup and reads resolvedCtes. The
    outermost statement's RTEs are always ctelevelsup 0 (KDoc at 888-892), so "if ctelevelsup == 0
    read own CTEs else read enclosing CTEs, with enclosing = empty" gives the same answer there.
    buildInnerCteNotNull and the inline block at 1305-1311 already are that rule.
  • Merge check. analyzeQueryBlockNullability handles a SubLink subselect or a FROM subquery.
    Neither can be a MERGE, so an empty mergeAbsentVarnos is equivalent.
  • DML qual gate. Same reasoning: a subselect is always a SELECT, so
    parseResultRelation(queryBlock) == 0 and the gate is a no-op there.
  • forceNewNullable. forcesNewNullable(SELECT block) is false, matching the current default.

Target state

One private class and one builder replace the three lambdas and two map-builders:

/** Everything needed to answer isSourceColumnNotNull for one query block. */
private class QueryBlockScope(
  val rangeTable: Map<Int, Int>,
  val hasGroupingSets: Boolean,
  val groupRteMap: Map<Pair<Int, Int>, Pair<Int, Int>>,
  val qualProvenVars: Set<Pair<Int, Int>>,
  val ownCtes: Map<String, List<Boolean>>,
  val enclosingCtes: Map<String, List<Boolean>>,
  val cteReferences: Map<Int, NodeTreeCteReference>,
  val subqueryColumnNotNull: Map<Pair<Int, Int>, Boolean>,
  val mergeAbsentVarnos: Map<Int, Boolean>,
  val forceNewNullable: Boolean,
) {
  fun isSourceColumnNotNull(varno: Int, varattno: Int, isColumnNotNull: (Pair<Int, Int>) -> Boolean): Boolean {
    if (mergeAbsentVarnos[varno] == true) return false
    if (isProvenByQuals(qualProvenVars, groupRteMap, varno, varattno)) return true
    rangeTable[varno]?.let { relid -> return isColumnNotNull(relid to varattno) }
    groupRteMap[varno to varattno]?.let { (baseVarno, baseAttno) ->
      val baseRelid = rangeTable[baseVarno] ?: return false
      return isColumnNotNull(baseRelid to baseAttno)
    }
    if (subqueryColumnNotNull[varno to varattno] == true) return true
    val reference = cteReferences[varno] ?: return false
    val ctesInScope = if (reference.ctelevelsup == 0) ownCtes else enclosingCtes
    return ctesInScope[reference.name]?.getOrNull(varattno - 1) == false
  }
}

private fun buildQueryBlockScope(
  queryBlock: String,
  enclosingCtes: Map<String, List<Boolean>>,
  applyQualNarrowing: Boolean,
  sql: String,
  depth: Int = SUBLINK_ANALYSIS_DEPTH_BUDGET,
  mergeAbsentVarnos: Map<Int, Boolean> = emptyMap(),
): QueryBlockScope

buildQueryBlockScope does, in order: parseRangeTable, hasGroupingSets, groupRteMap (empty
when grouping sets), resolveCteBodies(queryBlock, applyQualNarrowing, sql) for ownCtes,
buildSubqueryColumnNotNull(queryBlock, ownCtes, applyQualNarrowing, sql, depth),
parseCteRangeTableEntries, qualProvenNonNullVars gated on
applyQualNarrowing && !hasGroupingSets && parseResultRelation(queryBlock) == 0, and
forcesNewNullable(queryBlock).

buildAnalyzer gains an overload buildAnalyzer(scope: QueryBlockScope, depth: Int) that wires
scope::isSourceColumnNotNull (partially applied with ::isColumnNotNull), scope.hasGroupingSets,
scope.forceNewNullable, applyQualNarrowing, depth, and scope.ownCtes as resolvedCtes.

Then:

  • analyzeNodeTree builds one scope (enclosingCtes = emptyMap(), mergeAbsentVarnos from its
    parameter) and keeps only what is unique to it: the targetListByResno substitution and the
    RETURNING-first read. Its plainIsSourceColumnNotNull becomes
    { varno, varattno -> scope.isSourceColumnNotNull(varno, varattno, ::isColumnNotNull) }.
  • buildCteBodyAnalyzer becomes buildAnalyzer(buildQueryBlockScope(queryBlock, previouslyResolved, applyQualNarrowing, sql, mergeAbsentVarnos = mergeAbsentVarnos), depth = default).
  • analyzeQueryBlockNullability becomes buildAnalyzer(buildQueryBlockScope(queryBlock, resolvedCtes, applyQualNarrowing, sql, depth), depth).extractColumnNullability(queryBlock).
  • Delete buildCteColumnNotNull, buildInnerCteNotNull, and every "See analyzeNodeTree's identical
    guard" comment. The reasoning those comments carried (grouping-set null extension defeats qual
    narrowing; a DML body's WHERE can test a column its SET overwrites) moves once onto
    buildQueryBlockScope's KDoc.

isProvenByQuals stays as-is (it is already shared).

Test design

Behavior must not change. Pins:

  1. QueryAnalysisTest (545 live-PostgreSQL tests) and the golden scenarios are the primary pin.
  2. Before refactoring, confirm the three "incidental difference" claims above each have a covering
    test, and add one where missing, so the unification is proven rather than argued:
    • CTE referenced from inside a FROM subquery (ctelevelsup 1 in the subquery's own rtable):
      WITH c AS (SELECT id FROM t) SELECT s.id FROM (SELECT id FROM c) s with t.id NOT NULL
      id NOT NULL. Grep QueryAnalysisTest.kt for ctelevelsup; add under
      CommonTableExpressions if absent.
    • A nested WITH shadowing an enclosing CTE of the same name inside a CTE body
      (buildInnerCteNotNull's reason for existing). Add if absent.
    • A SubLink subselect whose body itself has a WHERE x IS NOT NULL narrowing (proves the qual
      gate still applies inside analyzeQueryBlockNullability). Add if absent.
  3. No new test may assert on private structure; all pins go through JdbcAnalyzer.analyzeQuery as the
    existing tests do.

Acceptance criteria

  1. ColumnNullabilityAnalyzer.kt contains exactly one isSourceColumnNotNull implementation and no
    buildCteColumnNotNull / buildInnerCteNotNull.
  2. The three tests above exist (pre-existing or added) and pass.
  3. ./gradlew :generator:check :gradle-plugin:test passes.
  4. Golden regeneration leaves test-scenarios* unchanged.
  5. ColumnNullabilityAnalyzer.kt is under 1000 lines.

Files

  • generator/src/main/kotlin/norm/generator/ColumnNullabilityAnalyzer.kt
  • generator/src/test/kotlin/norm/generator/QueryAnalysisTest.kt (additions only)

Conventions every issue inherits

  • Repo root: /Volumes/Code/3rd-party/norm. Module under change is almost always generator/.
  • Style: 2-space indent, 120 columns, full words in identifiers (parameter, not param), no
    section-separator comments, @Nested classes group tests. Format with ./gradlew spotlessApply.
  • TDD: tests are written first or alongside. For a behavior-preserving refactor the existing suite is
    the pin; each issue says which tests must also be added.
  • Golden files under test-scenarios*/ are never hand-edited. A refactor is behavior-preserving only
    if ./gradlew :gradle-plugin:generateGoldenFiles leaves git status --porcelain test-scenarios test-scenarios-frameworks empty.
  • Verification for a generator/ change: ./gradlew :generator:check :gradle-plugin:test (Docker
    required). Then the golden regeneration check above.
  • Commit message explains the design decision, not the diff.
  • Never run ./gradlew clean or disable the configuration/build cache to "fix" a build problem.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions