Skip to content

R4: pooled polling witness from rank screening and weak union (#197 unit 1) - #206

Merged
cameronfreer merged 13 commits into
masterfrom
r4-austin-polling
Aug 26, 2026
Merged

R4: pooled polling witness from rank screening and weak union (#197 unit 1)#206
cameronfreer merged 13 commits into
masterfrom
r4-austin-polling

Conversation

@cameronfreer

@cameronfreer cameronfreer commented Aug 25, 2026

Copy link
Copy Markdown
Owner

Unit 1 of the Austin route (#197; #107 remains open): the pooled polling witness, obtained from the assumed rank-n screening contract together with weak union.

This does not formalize Austin's Proposition 3.12. The construction follows Austin — a spare vertex reservoir, mixed clusters straddling it, and an enriched law that carries the polling data forward — but the conditional-independence engine is not his. RankRepresentation.screening is a field, an inductive hypothesis assumed at rank n, and its remainder already contains every other rank-≤ n block together with the whole pooled latent array. Weak union converts that directly into the mutual statement, so the tail-polling argument is not needed at this inductive stage and no tail machinery appears. The module says exactly this, and the now-unused polling-infrastructure import is dropped rather than a dependency manufactured.

The geometry

PoolVertex S s is Vinfinite S s ⊕ Vinfinite S s, with originalVertex = Sum.inl and poolVertex = Sum.inr, so the halves are disjoint definitionally and a PooledRankExtension is invariant under the full pooled permutation family.

The observed blocks are confined to the original half; the poll is confined to supports carrying at least one spare vertex. The two families are therefore disjoint by construction, which is the point — a block that could itself lie in the poll would make the conditional independence say nothing. Reading the blocks through the canonical pooledJointEquiv instead is tempting, because it makes the transport to C.P an exact identification, but it is wrong: that bijection sends blocks across both summands, and they would overlap the poll.

MixedClusterIndex means not wholly original — all-spare supports included, their original part being empty and hence proper. It is the negation of "wholly original", not a demand that both halves be met.

The seam is an enriched law

Forgetting the polling data into a bare C.P statement would discard what the successor construction consumes. enrichedPollingLaw retains the mixed clusters and the whole pooled latent array alongside the original structure and old latents, and enrichedPollingLaw_map_fst recovers C.P exactly, through Q.map_restrict_embedding at the original-vertex embedding — so the pooled gate theorem of #195 is a compiled dependency, verified by a transitive dependency walk with a control theorem that does not reach it.

The pooled latent array is carried because Q.screening's remainder contains it in full and weak union conditions on exactly that lower-rank factor. Every pooled index has cardinality < n, so the conditioning reveals no rank-n block.

The argument

  • condIndep_weak_unionif X ⊥ (Y, Z) | W and W ≤ Z then X ⊥ Y | Z. Neither Mathlib nor this repository had it. W ≤ Z is a hypothesis rather than a conclusion shape, so the result conditions on exactly Z, which is what keeps one fixed conditioning algebra across every stage of the peel.
  • restToPollingData — the screening remainder computes both the accumulated blocks and the entire polling conditioning, as a measurable factorization with an exact identity, so CondIndepFun.comp consumes Q.screening directly. This records what the remainder supplies rather than proving measurability coordinate-by-coordinate and discarding the assembled map.
  • polling_condExp_insert — the peel step as a direct conditional-expectation identity, so it drops into Finset.induction_on (with Finset.set_biInter_insert and Finset.prod_insert) and no dependent tuple reindexing happens inside the induction.
  • iCondIndepFun_originalBlock_sourcePollingCond — mutual conditional independence of the entire rank-n block family under Q.law. Mutual, not pairwise and not one-block-against-the-rest; the R4: successor contracts (interface-only) + shared regressions for both routes (#107) #196 battery exists to keep that distinction honest. Named at top level so it is independently reviewable.
  • pooledPollingWitness — pure transport. The block codomains genuinely differ, and blockSpaceCongr with enrichedBlock_comp_enrichedPollingMap relates them exactly, so no coercion is improvised inside the constructor.

Scope

Only the ambient Countable assumptions are needed. Because the conditional independence comes from the assumed screening contract rather than from a polling argument, the Fintype S.Srt hypothesis that the fixing-algebra stack carries is not required — it was inherited from the originally planned route and dropped once weak union replaced the tail engine. Rank zero is routed to nonempty_rankRepresentation_one with truncation_zero, respecting stepKernel's deliberate lack of an A = ∅ realization theorem. Nothing here asserts that the two routes' outputs agree, and no Kallenberg machinery appears.

mem_supportImage_iff is added to RelObservationGeometry: a third site where a classical DecidableEq used to form an image is not definitionally the natural instance available over a concrete carrier — here PoolVertex, whose Sum supplies one — so the membership form, carrying no instance in its type, is stated once alongside the geometry.

Bookkeeping

Audit 426 → 433 in both scripts/axiom_audit.lean and the intended-set literal in scripts/check_census_and_axioms.py. Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no warnings.

…xact export

Route A only. No Kallenberg machinery, and nothing here asserts the two routes'
outputs agree.

**The conditioning is pinned, not quantified.** `pollingObs` is a concrete
observation: the whole pooled rank-`n` latent array — every index of which has
cardinality `< n`, so it is proper-subset data carrying no rank-`n` block — together
with the structure read on the **spare** copy of the carrier, which is the poll.
An existential conditioning factor would let a witness condition on the whole joint
object and make the conclusion vacuous.

**The conclusion is mutual.** `PooledPollingWitness.mutualCondIndep` is
`iCondIndepFun` over the *entire* rank-`n` block family, which is Austin's
Proposition 3.12 conclusion. Pairwise independence, or one block against the rest,
would be strictly weaker — the #196 battery is what keeps that distinction honest.

**The export is not a pullback along a restriction.** `map_restrict_embedding`
pushes the pooled law forward onto `C.P` and is not invertible. The transport
therefore runs along the canonical equivalence's **inverse** measure-preserving map,
whose law identity is `map_poolVertexEquiv` — literally `Q.map_restrict_embedding`
at the canonical embedding. Verified mechanically: the gate theorem is transitively
reachable in the compiled dependency graph of `iCondIndepFun_blockMap`, and a
control theorem is not.

The block family is read through `pooledJointEquiv` rather than `originalVertex` so
that the export is an exact identification: composing two restrictions restricts
along the composite embedding, which is not the identity and would leave an
irreducible reindexing. The spare copy still enters, through `pollingObs`.

Scope: carries `[Fintype S.Srt]` to match the polling/fixing-algebra stack, leaving
the pooled API countable-only; rank zero is left to `nonempty_rankRepresentation_one`
with `truncation_zero`, respecting `stepKernel`'s lack of an `A = ∅` realization.

`iCondIndepFun_congr_cond` is private — the `CondIndepFun` form is already shared
glue, and this counterpart has one consumer.

Audit 426 → 429. Gates: lake build clean (3409 jobs), census + axiom audit pass,
zero sorries.
…ched law (#197)

Two corrections to the unit-1 spike, both substantive.

**The geometry was wrong.** Reading the block family through `pooledJointEquiv` made
the export to `C.P` an exact identification, and that convenience is what broke it:
`poolVertexEquiv` is a bijection `PoolVertex ≃ Vinfinite`, so blocks read through it
range across *both* summands rather than being confined to the original half, while
the poll read the spare half — observed blocks and poll were not disjoint, and a
block that can itself lie in the poll makes the conditional independence say nothing.
Blocks are now confined to `supportImage (originalVertex S)` (all `Sum.inl`) and the
poll to supports containing at least one `Sum.inr`, so the two families are disjoint
by construction.

**The seam is an enriched law, not bare `C.P`.** Austin's polling data is the family
of *mixed* clusters straddling the two halves — not merely the induced structure on
the spare half — and it must survive into the next law rather than being forgotten.
`enrichedPollingLaw` retains the clusters alongside the original structure and old
latents; `enrichedPollingLaw_map_fst` recovers `C.P` exactly, proved through
`Q.map_restrict_embedding (originalVertex S)`. Verified mechanically that the gate
theorem is transitively reachable in the compiled dependency graph, with a control
theorem that is not.

`MixedClusterIndex` makes the "indexed by proper original subsets" reading precise:
a rank-`n` support carrying a spare vertex has a proper original part.

Audit names updated to the surviving declarations. Gates: lake build clean (3409
jobs), census + axiom audit pass, zero sorries.

Unit 1 is **not** complete: no witness is constructed yet, so the polling engine in
`RelPollingInfrastructure` is still unconsumed. No PR until it is.
`MixedClusterIndex` means **not wholly original**, not literally straddling both
halves: an all-spare support qualifies, its original part being empty and therefore
proper. The prose claimed the stronger reading in four places, which would have
suggested the all-spare clusters are excluded when the definition includes them —
and they are exactly the clusters carrying no original data at all.

No statement or proof changes.
)

The peel permutations move original vertices into the spare half, which sends an
original latent index to a mixed one. The **original** latent array is therefore not
stable under those permutations, and any conditioning built from it cannot satisfy
the exact `comap ρ m₁ = m₂` identity the tail engine needs. The **pooled** array is
stable — such a permutation merely permutes pooled indices among themselves.

The pooled array is carried *alongside* the original marginal rather than replacing
it, so the first component of `enrichedPollingMap` is untouched and
`enrichedPollingLaw_map_fst` is literally unchanged. Every pooled index still has
cardinality `< n`, so the widened conditioning reveals no rank-`n` block and nothing
becomes vacuous.

Gates: lake build clean, census + axiom audit pass, zero sorries.
The graphoid axiom this route turns on, and which neither Mathlib nor this
repository had: **if X ⊥ (Y, Z) | W and W ≤ Z, then X ⊥ Y | Z.** The containment
W ≤ Z is a hypothesis rather than a conclusion shape, so the result conditions on
exactly Z and not on an unsimplified W ⊔ Z — which is what lets the peel keep one
fixed conditioning algebra across every stage.

This is what makes the tail engine unnecessary at the inductive stage.
`Q.screening` already holds at every pooled rank-n support, and its remainder is
(all other rank-≤n blocks, the whole pooled latent array) — which contains the
accumulated blocks, the clusters, and the latents. Weak union converts that
directly into the one-block statement, with the local conditioning absorbed
because it is measurable from the pooled-latent component.

Proof: enlarging the conditioning from W to any algebra between W and Y ⊔ Z leaves
the conditional probability of an X-event unchanged, by conditional-expectation
uniqueness with the product identity supplying the set integrals. Applying that at
Y ⊔ Z and at Z and peeling with the tower property gives the result.

Private, one consumer; it moves to ForMathlib when #198 needs it.

Two Lean notes worth keeping: abstract `MeasurableSpace Ω` binders shadow the
ambient instance, so ambient measurability is annotated `MeasurableSet[mΩ]` and the
cross-call instance is passed by name; and `f * S.indicator 1` is better handled as
`S.indicator f`, which makes integrability immediate instead of a bounded-mul
argument in the wrong factor order.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
…ceptance step 2a)

Three pieces, all about the *conditioning factor* rather than about what the
screening remainder supplies — deliberately committed before that more intricate
argument, so the two are separable if either needs rework.

* `sourcePollingCond` reads the polling conditioning directly on the pooled space,
  where steps 1–4 of the construction run.
* `pollingCond ∘ enrichedPollingMap = sourcePollingCond` holds by `rfl`: the
  source-level and enriched-level statements are the same observation, so relating
  them needs no transport.
* `comap_localLatents_le_sourcePollingCond` discharges weak union's `W ≤ Z`
  hypothesis concretely — the local conditioning at any pooled support is a
  coordinate projection of the pooled latent component, not an assumption.

Prose: the promotion note is now the standing rule (private at one consumer,
extracted after a second independent one) with no forward reference; and the
latent-widening justification rests on the durable reason — `Q.screening`'s
remainder contains the entire pooled latent array and weak union conditions on
exactly that lower-rank factor — rather than on peel-permutation stability, which
was true but is no longer the operative reason now that the tail engine is out of
the inductive stage.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
… acceptance steps 2b and 3)

**Check 2**, packaged as a measurable factorization rather than a bare algebra
inequality. `restToPollingData e F heF` sends the screening remainder to the
accumulated `F`-blocks paired with the entire polling conditioning, and
`restToPollingData_comp` proves the identity on the nose. This records exactly what
information `restObservationOver n A` supplies — the fact the induction depends on —
and lets `CondIndepFun.comp` consume `Q.screening` directly instead of rebuilding
measurability coordinate by coordinate and discarding the assembled map.

The two separations: `supportImage_ne_of_ne` reduces original-block-versus-`A`
equality to `B = e` by injectivity of `supportImage`, contradicting `e ∉ F`; and
`mixedCluster_ne_supportImage` uses that a cluster carries a spare vertex while the
image of an original support is wholly original.

**Check 3** is the named identity `comap_pollingCond_comp_enrichedPollingMap`, kept
explicit rather than left for `simp` to rediscover, so the forward descent at the end
of the construction is visibly exact.

`mem_supportImage_iff` is added to `RelObservationGeometry`: this is the third site
where a classical `DecidableEq` used to form an image is not definitionally the
natural instance available over a concrete carrier — here `PoolVertex`, whose `Sum`
supplies one — so image-shaped rewriting is unusable and the membership form, which
carries no instance in its type, is stated once alongside the geometry instead of
being worked around again locally.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
…isite)

`polling_condExp_insert`: the peel step, as a direct conditional-expectation
identity rather than a packaged conditional-independence statement, so that it drops
into `Finset.induction_on` with `Set.biInter_insert` and `Finset.prod_insert` and no
dependent tuple reindexing happens inside the induction.

Assembled from the pieces already banked: `Q.screening` at the original image of
`e`; its remainder composed with `restToPollingData`; `comap_prodMk` splitting the
resulting algebra into the accumulated blocks joined with the polling conditioning;
and `condIndep_weak_union` dropping back to the polling conditioning alone, with
check 1 supplying `W ≤ Z`.

The conclusion conditions on exactly `comap sourcePollingCond` — literally the same
term at every stage — which is what keeps the induction free of any accumulating
conditioning.

Lean note: `comap_prodMk` is stated against `MeasurableSpace.prod` while the goal
carries `Prod.instMeasurableSpace`; they are definitionally equal but not
syntactically, so the rewrite needs an explicit `rfl` bridge.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
…step 4)

`PooledRankExtension.iCondIndepFun_originalBlock_sourcePollingCond`: the whole
rank-`n` block family is mutually conditionally independent under `Q.law` given the
polling conditioning.

Named at top level rather than inlined into the witness, because it is the
load-bearing consumer of `polling_condExp_insert` and the direct input to the
forward descent. Keeping it separate makes it independently reviewable and leaves
the eventual witness constructor carrying no probability argument of its own.

The proof is structurally the singleton peel: rewrite by
`iCondIndepFun_iff_condExp_inter_preimage_eq_mul`, induct with `Finset.induction_on`,
discharge the empty stage by `condExp_const`, and discharge each insertion by
`polling_condExp_insert` against the induction hypothesis. Nothing accumulates in
the conditioning, since the insertion identity already conditions on exactly
`comap sourcePollingCond`.

Lean note: the index is a `Finset`, so the insert step needs
`Finset.set_biInter_insert`, not `Set.biInter_insert`.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
…ptance step 5)

`pooledPollingWitness` constructs a `PooledPollingWitness` for every pooled rank
extension. The constructor is **pure transport**: all probability content lives in
`iCondIndepFun_originalBlock_sourcePollingCond`, and step 5 only moves it along
`enrichedPollingMap`.

Two obligations, both discharged by named identities rather than coerced ad hoc:
the conditioning algebra, via `comap_pollingCond_comp_enrichedPollingMap`; and the
block **codomains**, which genuinely differ — the source family lands in
`BlockSpaceOver (PoolVertex S) (supportImage (originalVertex S) A)` while the witness
reads `blockMap A` on the restricted original structure. `blockSpaceCongr` is the
equivalence and `enrichedBlock_comp_enrichedPollingMap` the exact identity.

`iCondIndepFun_of_map` is brought in privately, adjacent to its one consumer, and
consumed immediately; the general form stays proved and preserved on its own branch
until a second independent consumer exists. `iCondIndepFun_comp` is the familywise
codomain composition Mathlib lacks, built from `Kernel.iIndepFun.comp` exactly as
`CondIndepFun.comp` is.

Closing items: the `RelPollingInfrastructure` import is **dropped**, since weak union
replaced the tail engine at this stage and manufacturing a dependency would be
dishonest; and the prose is recalibrated. The construction is Austin's — spare
reservoir, mixed clusters, enriched law — but the conditional-independence engine is
not: `screening` is a field assumed at rank `n`, weak union converts it, and
Proposition 3.12 is **not** reproved here. The witness docstring had claimed the
conclusion was Proposition 3.12 and is corrected.

Audit 429 → 433. Gates: lake build clean (3409 jobs), census + axiom audit pass, zero
sorries, no warnings.
)

The docstring referenced an unmerged branch as the home of the general form. An
unmerged branch is not part of the repository's API or provenance, and citing one is
the stale development-history prose we have been removing elsewhere. Replaced by the
timeless rule: private while it has one consumer, extract a general transport theorem
after a second independent consumer appears.

No statement or proof changes.
)

* **`[Fintype S.Srt]` removed.** It was inherited from the fixing-algebra stack this
  unit was originally planned to build on; once weak union replaced the tail engine
  nothing needed it, and `pooledPollingWitness` carries only the ambient `Countable`
  assumptions. The module scope note and the `Graphon.lean` entry now say so, with
  the reason rather than a bare correction.

* **Stale geometry.** "Straddling" is gone from both files: "mixed" is the negation
  of "wholly original", so all-spare clusters are included. `pollingCond` is
  described as the whole pooled rank-`n` latent array plus the mixed clusters, not
  old or original latents. The permutation-stability rationale on `pollingCond` is
  replaced by the durable one — `Q.screening`'s remainder contains the pooled array
  in full and weak union conditions on exactly that lower-rank factor. That rationale
  had been corrected once at the header; this was a second copy on the definition.

* **Warnings cleared.** `pooledPollingWitness` is a `theorem`, not a Prop-valued
  `def`; six theorems gained `omit` clauses — four flagged plus two that surfaced
  once the `Fintype` variable was gone. `omit ... in` must precede the doc comment.

No statement or proof changes beyond the removed hypothesis, which strictly
generalizes the public signature.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
Three residues from the widening that carried the pooled rank-`n` latent array into
the enriched data. The `EnrichedSpace` docstring and `pollingCond`'s rationale were
updated at the time; these summaries of the same objects were not:

* `pollingCond` is the whole pooled rank-`n` latent array together with the clusters,
  not the old latents;
* the enriched-law summaries in the module header, on `enrichedPollingLaw`, and in
  the `Graphon.lean` entry now name the pooled array alongside the clusters;
* `Set.biInter_insert` corrected to `Finset.set_biInter_insert` in the insertion
  identity's docstring.

No statement or proof changes.

Gates: lake build clean (3409 jobs), census + axiom audit pass, zero sorries, no
warnings.
@cameronfreer
cameronfreer merged commit daebbe2 into master Aug 26, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant