perf(linalg): opt-in solve_ols normal-equations Cholesky fast path (DIFF_DIFF_SOLVE_OLS_FASTPATH)

[igerber]

Summary

• Opt-in DIFF_DIFF_SOLVE_OLS_FASTPATH=1 normal-equations Cholesky fast path for solve_ols, both backends. After perf(linalg): per-cell solver fast paths - IRLS Cholesky inner solve + rank-detection certification (CS covariate fits 1.5-2x) #634's stage-0 Gram certification removed the pivoted-QR cost, the unconditional SVD solve is the dominant stage on solver-bound fits (58%/54%/40% of SunAbraham county/firm and CallawaySantAnna dr cov40 fit time). The knob routes certified-well-conditioned full-rank solves through an equilibrated normal-equations Cholesky; any certification decline falls through the UNCHANGED legacy chain verbatim.
• numpy twin: reuses the stage-0 rank-certification artifacts (the Gram is already built to certify rank), gated by cho_factor + LAPACK dpocon at reciprocal condition > 1e-6 (the _IRLS_CHOL_RCOND_GUARD error budget, ~eps*cond <= 2e-10); self-builds and self-certifies on the skip_rank_check route.
• Rust kernel solve_ols_chol: separate self-certifying pyfunction (stale extensions degrade via the independent-import pattern): faer Llt, exact 1-norm rcond whose inverse byproduct doubles as the sandwich bread, one owned n x k buffer reused in place for the vcov scores (no thin-SVD U transient), faer-only heavy ops so the accelerate/openblas/no-BLAS wheel variants behave identically, shared first-appearance cluster indexer with the SVD path.
• Measured (M4 Max, medians; committed A/B lane benchmarks/speed_review/bench_solve_ols_fastpath.py with a pristine-base-tree identity gate): SunAbraham 1.13->0.63 s / 16.77->9.89 s (1.7-1.8x), CS dr 40-covariate 4.40->3.57 s (1.23x), skip_rank_check micro-solve 1.56x, weighted/survey twin 1.07x; estimate deltas <= 9e-15 ATT / 3e-12 rel SE; rust allocator high-water on the 2.4Mx130 clustered solve 7.27 -> 2.66 GB. Knob-off ATT/SE byte-identical to the base tree on every scenario.
• Also fixed: the legacy Rust SVD vcov path silently returned all-Inf vcov on saturated full-rank designs (n == k) — the (n-k) adjustment divides by zero and the dispatcher fallback check keyed on NaN only. Both Rust vcov paths now return the documented all-NaN sentinel (matching the canonical numpy saturated guard; G >= 2 cluster error keeps precedence), and the dispatcher rejects any non-finite Rust vcov via a shared helper (on the skip_rank_check route only Inf reroutes — all-NaN remains the documented sentinel there). This is the sole, CHANGELOG-documented exception to the default-path byte-identity claim.

Methodology references (required if estimator / math changes)

• Method name(s): N/A — no estimand or estimator change. Numerical solve-algorithm alternative (equilibrated normal-equations Cholesky vs SVD/gelsd) on an opt-in path, certification-gated; plus the non-finite-inference contract fix (all-NaN vcov on zero residual df).
• Paper / source link(s): N/A (numerical linear algebra; thresholds documented in REGISTRY.md's rank-deficiency/backend-parity Note cluster with the certification-nesting rationale).
• Any intentional deviations from the source (and why): Within the opt-in path, parity vs the default solve is tol-bounded, not bit-level (fitted ~1e-8 abs / SE ~1e-6 rel; documented REGISTRY Note). Default OFF stays byte-identical except the saturated n == k vcov sentinel fix (Inf -> NaN + warning), which aligns Rust with the canonical numpy contract.

Validation

• Tests added/updated: tests/test_linalg.py (resolver conventions; property parity across ~20 designs; forced-fallback and natural-guard-trip byte-identity with a self-checking BETWEEN fixture; default-path spy locks; skip_rank_check self-certification decline; WLS incl. zero weights; rank-deficient interplay; cert-artifact reuse bitwise equality; hc2/conley spot checks), tests/test_rust_backend.py (kernel-vs-twin parity at the established decimal bars; 20-call clustered bit-determinism; decline/None contracts; dispatch spies proving the default path unchanged; stale-symbol degradation; estimator-level SunAbraham + DiD-absorb-cluster parity; saturated-contract classes for both kernels; Inf-injection reroutes on both routes; NaN-sentinel pass-through), plus 7 new cargo tests (LU oracles, decline contracts, saturated NaN, G<2 precedence).
• Backtest / simulation / notebook evidence (if applicable): committed before/after benchmark baselines under benchmarks/speed_review/baselines/solve_ols_fastpath_{before,after}.json; results tables in docs/performance-plan.md.

Security / privacy

• Confirm no secrets/PII in this PR: Yes

🤖 Generated with Claude Code

https://claude.ai/code/session_01GPX5Rv8ozQXPdUV23QTfjr

[github-actions]

Overall Assessment

✅ Looks good — no unmitigated P0/P1 findings.

Executive Summary

• Affected methods: shared solve_ols, HC1/CR1 robust vcov paths, and downstream estimators that route through solve_ols.
• The opt-in Cholesky fast path is explicitly documented in docs/methodology/REGISTRY.md:L684-L689; this is an implementation choice, not an undocumented methodology change.
• The saturated n == k Rust vcov fix aligns with the documented NaN non-finite inference contract in REGISTRY.md:L690-L695.
• I did not find new inline inference computation, partial NaN guards, or missing parameter propagation that would create misleading statistical output.
• I could not run Python tests because this sandbox lacks numpy; cargo test could not fetch crates due blocked network/DNS.

Methodology

• Severity: P3 informational
  Impact: The new DIFF_DIFF_SOLVE_OLS_FASTPATH changes the numerical solve algorithm for certified full-rank solve_ols fits, but the deviation is documented as opt-in and tol-bounded in docs/methodology/REGISTRY.md:L684-L689.
  Concrete fix: None required.

• Severity: P3 informational
  Impact: The Rust saturated-design fix returns all-NaN vcov instead of Inf for undefined residual df, consistent with the registry’s non-finite inference contract at docs/methodology/REGISTRY.md:L690-L695 and the NumPy guard in diff_diff/linalg.py:L3490-L3497.
  Concrete fix: None required.

Code Quality

• Severity: P3 informational
  Impact: The new routing is scoped and guarded: Rust Cholesky is limited to unweighted HC1/CR1, while weighted/non-HC1 paths use the NumPy twin and recompute weighted vcov on original-scale data in diff_diff/linalg.py:L1413-L1476.
  Concrete fix: None required.

Performance

• Severity: P3 informational
  Impact: The performance change is opt-in and the default-on follow-up is tracked in TODO.md:L42-L45, so this PR does not silently alter default numerical behavior.
  Concrete fix: None required.

Maintainability

• Severity: P3 informational
  Impact: The Rust Cholesky symbol is independently imported for stale-extension degradation in diff_diff/_backend.py, and the dispatcher falls back cleanly when unavailable.
  Concrete fix: None required.

Tech Debt

• Severity: P3 informational
  Impact: The remaining default-flip work is explicitly tracked in TODO.md:L42-L45. This is acceptable deferred work under the review rules.
  Concrete fix: None required.

Security

• Severity: P3 informational
  Impact: No secrets, credentials, or risky external input handling were found in the changed code or benchmark baselines.
  Concrete fix: None required.

Documentation/Tests

• Severity: P3 informational
  Impact: The PR adds registry, changelog, performance-plan, Python, and Rust test coverage for resolver behavior, parity, fallback, stale-symbol degradation, saturated NaN vcov, and Inf rerouting. I could not execute the full checks locally: Python import failed because numpy is unavailable, and cargo test failed because crates.io was unreachable.
  Concrete fix: None for the PR; rely on CI in an environment with Python deps and Cargo registry access.