Group generated renderers into hash-assigned source files for incremental compilation

CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## Unreleased
+
+- Generate renderers into a fixed set of 32 source files chosen by a stable hash of the
+  page-model name, each with its own static resource, instead of one source file holding
+  every renderer. Renderers no longer reference the registry, so an unchanged file
+  regenerates byte-identical output and Gradle's incremental Java compilation recompiles
+  only the file whose template changed plus the registry. In ReAI a one-template edit now
+  takes 4.3 s end to end instead of 10.4–12.1 s (`compileJava` 6.2 s → 0.3 s, `compileKotlin`
+  1.1 s → 0.09 s), and a full `compileJava` takes 2.8 s. Rendered HTML is byte-identical to
+  0.11.2, checked by recorded golden renders. Generated renderer classes are now nested in
+  `<Registry>Part<n>` holders and end in `Renderer`; they are package-private, not API.
+  Static content is deduplicated per file rather than per module, so ReAI's resources grow
+  from 1.6 MB to 2.8 MB.
+
 ## 0.11.2
 
 - Dispatch generated template registries through a class index instead of linear `==` and

DESIGN.md

@@ -14,7 +14,7 @@ Thim optimizes for four properties:
 1. The Gradle plugin tracks HTML, strict YAML message catalogs and model sources.
 2. KSP resolves a page-model class from the template filename and configured model packages.
 3. The compiler links fixed layouts and fragments, then validates properties, nullability, locale/key/argument parity, plural and select rules, and supported directives.
-4. It emits readable Java renderers and one package-local resource containing static UTF-8 content.
+4. It emits readable Java renderers grouped into a fixed set of source files by a stable hash of the page-model name, each file with its own package-local resource of static UTF-8 content, plus a registry that maps page-model classes to renderers. A renderer depends only on its file's resource, so an unchanged file regenerates byte-identical output and Gradle's incremental Java compilation skips it.
 5. Each template jar publishes its generated registry through Java's service loader.
 
 Kotlin modules use the KSP Gradle integration. Java modules run KSP2 directly against Java sources, including records and bean accessors. Both paths call the same compiler and generate the same runtime code.

PERFORMANCE_AUDIT.md

@@ -98,7 +98,7 @@ and allocations are recorded in [the benchmark results](benchmark/results/2026-0
 
 | Priority | Finding | Next step |
 | --- | --- | --- |
-| Highest follow-up | Generated renderers share one Java source and static resource, with aggregating KSP dependencies. ReAI's existing local output contains 314 renderer classes in a 13.49 MB source file, plus a 1.51 MB static resource. | Profile a one-model edit and a one-template edit. Consider per-template sources/resources with isolating dependencies, while retaining a separate aggregating registry. Shared layouts, message changes, additions, and deletions need explicit invalidation coverage. |
+| Highest follow-up | Generated renderers share one Java source and static resource, with aggregating KSP dependencies. ReAI's existing local output contains 314 renderer classes in a 13.49 MB source file, plus a 1.51 MB static resource. | Implemented as per-template sources and resources after 0.11.2; see the per-template section below. |
 | Medium | Each expression property lookup can walk KSP properties and supertypes again, including properties from shared layouts. | Profile symbol-resolution time, then consider a cache confined to one processor invocation. Keep missing-property diagnostics and generic/inherited property behavior intact. |
 | Medium | Generated `supports`, `supportsReturnType`, request-data checks, and render dispatch use linear checks. | Add a benchmark with hundreds of page models and the full Spring handler path before replacing dispatch with a map or `ClassValue`. Preserve custom `TemplateSet` behavior and subclass handling. |
 | Medium | Servlet rendering allocates an 8 KiB body buffer and a 1 KiB output buffer, then buffers the whole response. | Measure full pages and small HTMX responses through the Spring adapter. Tune sizing only with allocation and latency evidence. Streaming changes failure handling and content-length behavior, so it needs separate design work. |
@@ -342,3 +342,51 @@ long plain-ASCII runs and loses on the text these applications render:
 
 The current per-character loop runs at about 0.6 ns per character, close to the copy
 floor, and stays.
+
+## Grouped generated sources — 18 September 2026
+
+After 0.11.2 the compiler emits renderers into 32 source files chosen by
+`floorMod(modelName.hashCode(), 32)`, each with a holder class that owns the file's static
+resource; renderers are nested in that holder and reference only its `STATIC`. The registry
+is the only file that references every renderer. KSP dependencies stay aggregating, so KSP
+regenerates everything on each run; the gain comes from Gradle's incremental Java
+compilation seeing byte-identical files for unchanged templates, and from Kotlin's
+incremental compilation seeing one changed Java source instead of one huge one.
+
+Two intermediate layouts were measured and rejected on ReAI (344 renderers):
+
+| Layout | Forced `kspKotlin` | Edit `compileJava` | Resources | Note |
+| --- | ---: | ---: | ---: | --- |
+| 0.11.2, one source | 2.8–3.0 s | 6.1–6.3 s | 1.6 MB | baseline |
+| one file per template | 4.4–4.7 s | 0.26–0.28 s | 6.5 MB | KSP registers 345 generated Java files: Thim's own generation grew only 0.07 → 0.30 s, the rest is KSP; no static dedup |
+| 32 hash-assigned files | 3.6 s | 0.29–0.31 s | 2.8 MB | shipped |
+
+The whole edit loop (`:web-app:classes` after touching one template, warm daemon, build
+cache disabled, two or three repetitions):
+
+| Task | 0.11.2 | 32 files |
+| --- | ---: | ---: |
+| `:web-app:kspKotlin` | 3.0–4.0 s | 3.6 s |
+| `:web-app:compileKotlin` | 1.0–1.2 s | 0.09 s |
+| `:web-app:compileJava` | 6.1–6.3 s | 0.3 s |
+| Total build | 10.4–12.1 s | 4.3–4.4 s |
+
+Full `:web-app:compileJava` fell from 6.3 s to 2.8 s. Processor phases were timed with
+temporary instrumentation: parsing, expansion, catalog, strict-model checks, routes and
+compilation take 2.1–2.6 s in every layout, so the remaining KSP task time is KSP's own
+handling of generated files.
+
+Two details matter for the upgrade. First, the renderer classes are named `…Renderer`
+instead of `…ThimRenderer`: with the old names, Gradle's previous-compilation data still
+listed every renderer under `ThimTemplates.java` after the layout change, so the next
+one-template edit deleted all class files while passing two sources to javac, and retrying
+did not recover. Second, renderers are nested in their holder rather than declared as
+package-private top-level classes, because javac's auxiliary-class lint rejects those when
+referenced from another file and one ReAI module compiles with `-Werror`. With both in
+place, the sequence 0.11.2 full build → new layout (incremental) → edit → retry → second
+edit → revert succeeds with the build cache disabled; Gradle's reported class counts for
+an edit include stale old names it deletes as no-ops in about 0.25 s.
+
+Output equivalence is checked by `GoldenRenderTest` in the example module: twelve renders
+(four pages, three locales, including form errors and a 64-byte output buffer) recorded
+with the 0.11.2 compiler and compared byte for byte.

compiler/src/main/kotlin/no/beint/thim/compiler/RendererGenerator.kt

@@ -9,6 +9,10 @@ import com.google.devtools.ksp.symbol.Nullability
 import java.io.ByteArrayOutputStream
 import java.nio.charset.StandardCharsets
 
+/**
+ * One renderer class. Its static UTF-8 content lives in the shared [StaticContent] of the
+ * generated source file it is grouped into, referenced through that file's holder class.
+ */
 internal data class CompiledTemplate(
     val model: KSClassDeclaration,
     val rendererName: String,
@@ -32,8 +36,6 @@ internal class StaticContent {
 internal class RendererGenerator(
     private val catalog: MessageCatalog,
     private val routes: RouteCatalog,
-    private val staticContent: StaticContent,
-    private val registryName: String,
     private val strictModels: Boolean = false,
 ) {
     private var generatedVariable = 0
@@ -49,10 +51,25 @@ internal class RendererGenerator(
     fun usedRootProperties(model: KSClassDeclaration): Set<String> =
         usedRootProperties[model.qualifiedName?.asString()] ?: emptySet()
 
-    fun compile(templateName: String, model: KSClassDeclaration, nodes: List<Node>): CompiledTemplate {
+    /**
+     * [staticContent] and [holderName] belong to the generated source file this renderer is
+     * grouped into: static runs are appended to that file's shared content and referenced as
+     * `holderName.STATIC`.
+     */
+    fun compile(
+        templateName: String,
+        model: KSClassDeclaration,
+        nodes: List<Node>,
+        staticContent: StaticContent,
+        holderName: String,
+    ): CompiledTemplate {
         val modelName = model.qualifiedName?.asString() ?: error("$templateName: model must have a qualified name")
-        val rendererName = modelName.replace(Regex("[^A-Za-z0-9_]"), "_") + "ThimRenderer"
-        val code = CodeWriter(staticContent, registryName)
+        // The suffix differs from the single-file layout's "ThimRenderer" on purpose: Gradle's
+        // incremental Java compilation keeps the registry file's stale class list across the
+        // layout change, and reusing those class names made it delete every renderer class
+        // while recompiling only the edited one.
+        val rendererName = modelName.replace(Regex("[^A-Za-z0-9_]"), "_") + "Renderer"
+        val code = CodeWriter(staticContent, holderName)
         val locales = if (usesMessages(nodes)) {
             catalog.supportedLocales.filterTo(linkedSetOf()) { it != catalog.defaultLocale }
         } else {
@@ -73,7 +90,9 @@ internal class RendererGenerator(
         generatedHelper = 0
         pendingHelpers.clear()
 
-        code.line("final class $rendererName {")
+        // Nested in its file's holder class: javac's auxiliary-class lint (fatal under -Werror)
+        // rejects package-private top-level classes referenced from another source file.
+        code.line("static final class $rendererName {")
         code.indent {
             code.line("private $rendererName() {}")
             code.line()
@@ -1422,7 +1441,7 @@ internal class RendererGenerator(
 
     private class CodeWriter(
         private val staticContent: StaticContent,
-        private val registryName: String,
+        private val holderName: String,
     ) {
         private val output = StringBuilder()
         private val pending = StringBuilder()
@@ -1464,7 +1483,7 @@ internal class RendererGenerator(
             if (pending.isEmpty()) return
             val range = staticContent.append(pending.toString())
             output.append("    ".repeat(depth))
-                .append("output.raw($registryName.STATIC, ")
+                .append("output.raw($holderName.STATIC, ")
                 .append(range.first)
                 .append(", ")
                 .append(range.last - range.first + 1)

compiler/src/main/kotlin/no/beint/thim/compiler/ThimProcessor.kt

@@ -110,10 +110,11 @@ private class ThimProcessor(
             } else {
                 RouteCatalog(emptyList(), emptyList(), extractedRoutes.files)
             }
-            val staticContent = StaticContent()
-            val generator = RendererGenerator(catalog, routeCatalog, staticContent, registryName, strictModels)
+            val generator = RendererGenerator(catalog, routeCatalog, strictModels)
+            val statics = List(RENDERER_FILES) { StaticContent() }
             val compiled = templates.map { template ->
-                generator.compile(template.name, template.model, template.nodes)
+                val file = rendererFile(template.model)
+                generator.compile(template.name, template.model, template.nodes, statics[file], holderName(file))
             }
             problems += generator.errors
             collect(problems) { if (strictModels) reportUnusedProperties(templates, generator) }
@@ -126,7 +127,7 @@ private class ThimProcessor(
                 completed = true
                 return emptyList()
             }
-            generate(compiled, staticContent.bytes(), extractedRoutes)
+            generate(compiled, statics, extractedRoutes)
             if (catalog.definitions().isNotEmpty()) {
                 val files = (compiled.mapNotNull { it.model.containingFile } + extractedRoutes.files).distinct().toTypedArray()
                 generateMessages(catalog, files)
@@ -188,15 +189,49 @@ private class ThimProcessor(
         diagnostic("THIM-MODEL-UNUSED-PROPERTY", null, unused.joinToString("; "))
     }
 
-    private fun generate(compiled: List<CompiledTemplate>, staticContent: ByteArray, routeCatalog: RouteCatalog) {
+    /**
+     * Renderers are grouped into a fixed number of source files by a stable hash of the
+     * page-model name, each with its own static resource. A renderer references only its
+     * file's holder class, never the registry or another file, so an unchanged file
+     * regenerates byte-identical output and Gradle's incremental Java compilation recompiles
+     * only the file whose template changed plus the registry. A fixed count keeps KSP's
+     * per-file overhead small; membership depends on the model name alone, so adding or
+     * removing a template touches one file.
+     */
+    private fun generate(compiled: List<CompiledTemplate>, statics: List<StaticContent>, routeCatalog: RouteCatalog) {
         val files = (compiled.mapNotNull { it.model.containingFile } + routeCatalog.files).distinct().toTypedArray()
         val dependencies = Dependencies(aggregating = true, *files)
-        codeGenerator.createNewFile(
-            dependencies = dependencies,
-            packageName = generatedPackage,
-            fileName = registryName,
-            extensionName = "bin",
-        ).use { it.write(staticContent) }
+        compiled.groupBy { rendererFile(it.model) }.toSortedMap().forEach { (file, members) ->
+            val holder = holderName(file)
+            codeGenerator.createNewFile(
+                dependencies = dependencies,
+                packageName = generatedPackage,
+                fileName = holder,
+                extensionName = "bin",
+            ).use { it.write(statics[file].bytes()) }
+            codeGenerator.createNewFile(
+                dependencies = dependencies,
+                packageName = generatedPackage,
+                fileName = holder,
+                extensionName = "java",
+            ).bufferedWriter(StandardCharsets.UTF_8).use { output ->
+                output.appendLine("package $generatedPackage;")
+                output.appendLine()
+                output.appendLine("import java.io.IOException;")
+                output.appendLine("import no.beint.thim.HtmlOutput;")
+                output.appendLine("import no.beint.thim.RenderContext;")
+                output.appendLine()
+                output.appendLine("final class $holder {")
+                output.appendLine("    static final byte[] STATIC = HtmlOutput.resource($holder.class, \"$holder.bin\");")
+                output.appendLine()
+                output.appendLine("    private $holder() {}")
+                members.sortedBy { it.rendererName }.forEach { template ->
+                    output.appendLine()
+                    output.append(template.source.prependIndent("    ").replace(Regex("(?m)^ +$"), ""))
+                }
+                output.appendLine("}")
+            }
+        }
         codeGenerator.createNewFile(
             dependencies = dependencies,
             packageName = generatedPackage,
@@ -210,10 +245,7 @@ private class ThimProcessor(
             output.appendLine("import no.beint.thim.RenderContext;")
             output.appendLine("import no.beint.thim.TemplateSet;")
             output.appendLine()
-            compiled.forEach { output.append(it.source).appendLine() }
             output.appendLine("public final class $registryName implements TemplateSet {")
-            output.appendLine("    static final byte[] STATIC = HtmlOutput.resource($registryName.class, \"$registryName.bin\");")
-            output.appendLine()
             output.appendLine("    // Exact page-model classes resolve in constant time; the instanceof chain in render only")
             output.appendLine("    // serves subclasses of open models, matching the previous linear dispatch.")
             output.appendLine("    private static final java.util.Map<Class<?>, Integer> INDEX = index();")
@@ -264,7 +296,7 @@ private class ThimProcessor(
             output.appendLine("            switch (index) {")
             compiled.forEachIndexed { index, template ->
                 val modelName = template.model.qualifiedName!!.asString()
-                output.appendLine("                case $index -> ${template.rendererName}.render(($modelName) model, context, output);")
+                output.appendLine("                case $index -> ${rendererReference(template)}.render(($modelName) model, context, output);")
             }
             output.appendLine("                default -> throw new IllegalStateException(\"Unknown template index \" + index);")
             output.appendLine("            }")
@@ -273,7 +305,7 @@ private class ThimProcessor(
             compiled.forEach {
                 val modelName = it.model.qualifiedName!!.asString()
                 output.appendLine("        if (model instanceof $modelName typed) {")
-                output.appendLine("            ${it.rendererName}.render(typed, context, output);")
+                output.appendLine("            ${rendererReference(it)}.render(typed, context, output);")
                 output.appendLine("            return;")
                 output.appendLine("        }")
             }
@@ -384,7 +416,18 @@ private class ThimProcessor(
     private fun requiredPath(environment: SymbolProcessorEnvironment, key: String): Path =
         Path.of(requireNotNull(environment.options[key]) { "Missing KSP option '$key'" }).toAbsolutePath().normalize()
 
+    private fun rendererFile(model: KSClassDeclaration): Int =
+        Math.floorMod(model.qualifiedName!!.asString().hashCode(), RENDERER_FILES)
+
+    private fun holderName(file: Int): String = "${registryName}Part$file"
+
+    private fun rendererReference(template: CompiledTemplate): String =
+        "${holderName(rendererFile(template.model))}.${template.rendererName}"
+
     private companion object {
+        /** Generated renderer source files per module; see [generate]. */
+        const val RENDERER_FILES = 32
+
         fun conventionalModelName(templateName: String): String = templateName
             .split(Regex("[^A-Za-z0-9]+"))
             .filter(String::isNotEmpty)

example/build.gradle.kts

@@ -27,6 +27,7 @@ dependencies {
 
 tasks.test {
     useJUnitPlatform()
+    systemProperty("thim.golden.record", System.getProperty("thim.golden.record", "false"))
 }
 
 ksp {

example/src/test/java/no/beint/thim/example/GeneratedRendererTest.java

@@ -74,7 +74,10 @@ private static String encoded(String value) {
 
     @Test
     void partitionsLargeRenderersBelowTheHotSpotHugeMethodThreshold() throws IOException {
-        var resource = "/no/beint/thim/example/generated/no_beint_thim_example_page_LargePageThimRenderer.class";
+        // Renderers nest in a holder chosen by the same stable hash the compiler uses.
+        var file = Math.floorMod(LargePage.class.getName().hashCode(), 32);
+        var resource = "/no/beint/thim/example/generated/ExampleTemplatesPart" + file
+                + "$no_beint_thim_example_page_LargePageRenderer.class";
         byte[] bytes;
         try (var input = getClass().getResourceAsStream(resource)) {
             bytes = input.readAllBytes();