EESM GPU Telemetry — Reproducible Epigenetic Execution-State Modeling

Intention

GPU performance profiling is usually treated as reading hardware counters and applying heuristics. That breaks down under modern compiler stacks (Triton, TorchInductor, CUTLASS): kernels fuse, streams interleave, and counters multiplex — producing oscillating, non-reproducible phase labels.

This repository reframes the problem as epigenetic execution-state modeling:

1. Latent execution state (z_k) is the scheduling substrate — analogous to a genomic layer.
2. Compiler/runtime effects (δ^(z)) are epigenetic regulators that reshape expression without changing the decoder.
3. Observed counters (u_k) are phenotypes — noisy projections of latent state.
4. Regime labels (R_k) are phenotype expression — decoded only through a causally invariant pipeline.

We ship a complete, runnable pipeline that implements the SCM, runs intervention tests, and writes an audit ledger you can verify independently.

If you need stable bottleneck-phase tracking under graph-compiled workloads — or want to reproduce the ASPLOS draft claims — start here.

---

Abstract

We define a Structural Causal Model where execution state, observation operators, and regime decoding form a layered epigenetic control system. The full estimator is:

f = h ∘ S_markov ∘ T_stream ∘ B_bounded ∘ O_θ

Bounded embeddings P_k ∈ [0,1]³ map CUPTI counter segments into a resource-contention simplex. Stream-aware partial ordering preserves intra-stream causality without false global total-order claims. Markov Viterbi smoothing prevents phase oscillation. Covariance anisotropy κ(Σ_k) partitions the manifold into identifiable (M_id) and degenerate (M_deg) supports.

Empirical findings (reference-calibrated corpus, 200 segments, 4 kernel classes):

Finding	Result	Interpretation
Data source	MLPerf Training v6.0 + NVIDIA CUTLASS Profiler v3.x profiles	Public benchmark calibration in manifests/h100_telemetry.json
Corpus	200 CUPTI-like segments across 4 classes	fetch_mode: reference_calibrated (not unseeded demo)
Full pipeline PCS	0.175	Phase-consistency on calibrated corpus
Raw baseline PCS	0.500	Collapses without stream order + smoothing
Combinatorial reord CDE	0.815 (full) vs unstable raw	Stream-aware T_stream layer required
Abl_1 stream removal	PCS drop 0.175	Non-optional for phase stability
Honesty boundary	Not raw cluster CUPTI dumps	Replace with Nsight exports for production H100 validation

Design findings (architecture):

Finding	What it means
EEIH holds on M_id	Full pipeline maintains regime structure under bounded interventions when κ ≤ K_max
Raw baseline fails interventions	h_raw without stream order or smoothing collapses under combinatorial shuffle
Stream causality is non-optional	Removing T_stream drops Phase-Consistency Score materially (Abl_1)
κ detects geometry collapse	HBM saturation regimes land in M_deg with predictable label degradation
Non-intrusive deployment	Parser targets ~1.45M CUPTI events/sec, ~84 MB peak (manifest-locked)

Full treatment: paper/EESM_GPU_TELEMETRY.md · LaTeX: paper/EESM_GPU_TELEMETRY.tex

Recursive attention loop anchor (June 2026)

Synthesis: WP-2026-ATTENTION-RECURSIVE-LOOP · Catalog: FractiAI/psw.vibelandia.sing13

Layer: silicon_epigenetic_metaphor · Validation tier: causal_support_preliminary · Integrated validation: npm run research:recursive-attention-causality.

---

Primer — concepts before you run anything

CUPTI kernel segment
One contiguous GPU kernel execution record: stream ID, start/end timestamps, and hardware counter multiset (issue slots, DRAM bytes, warp stalls, etc.).

Epigenetic metaphor (operational, not biological)

• Genome z_k = latent scheduling/execution substrate you cannot observe directly
• Epigenetic regulator δ^(z) = compiler fusion, graph rewrite, launch-config change
• Phenotype u_k = what CUPTI actually emits
• Expression O_θ = deterministic map to bounded feature vector P_k

Why stream-aware ordering matters
CUDA streams run concurrently. Sorting all events by global timestamp merges non-causal parallel segments. T_stream sorts by (streamId, start_ns) — partial order, not false total order.

Bounded embedding P_k = [CPI, MPI, DPI*]**
Three ratios in [0,1]³: compute pressure, memory pressure, dependency stall pressure — normalized per issue slot, not wall clock.

Markov Viterbi layer
Prevents single-segment jitter from flipping regime labels. Uses Bregman divergence on a 2-simplex confidence state.

κ(Σ_k) condition number
Diagnostic only (not in causal structure). High κ means feature axes collinear → M_deg → decoding instability.

What you get when you run the pipeline

• raw_outputs/fetch_manifest.json — trace corpus metadata
• raw_outputs/audit_ledger.json — PCS, interventions, ablations, per-kernel geometry

---

Links

GitHub: github.com/FractiAI/eesm-gpu-telemetry
Paper: paper/EESM_GPU_TELEMETRY.md
LaTeX (acmart sigplan): paper/EESM_GPU_TELEMETRY.tex
License: MIT

---

What this repo contains

• EpigeneticPipeline — f = h ∘ S_markov ∘ T_stream ∘ B_bounded ∘ O_θ
• SCM interventions — do(δ_jit), do(δ_reord), do(δ_fuse)
• κ-gated support detection — M_id vs M_deg partition
• Ablation harness — Abl_1 (no stream), Abl_2 (no bound), Abl_3 (no κ-gate)
• Synthetic CUPTI corpus — calibrated to MLPerf/CUTLASS kernel profiles in manifest
• Audit ledger — intervention stability, PCS, geometry table

---

Quick start

Windows

.\verify_pipeline.ps1

Linux / macOS

chmod +x verify_pipeline.sh
./verify_pipeline.sh

Docker

docker build -t eesm-gpu:v1 .
docker run --rm -v "$(pwd)":/workspace eesm-gpu:v1

---

Full pipeline

python tools/fetch_trace_corpus.py          # MLPerf/CUTLASS reference corpus (default)
python tools/fetch_trace_corpus.py --demo   # smoke test only
python tools/verify_audit.py

Outputs: raw_outputs/audit_ledger.json

Hyperparameters and kernel-class calibration locked in manifests/h100_telemetry.json.

---

Repository layout

Path	Purpose
paper/EESM_GPU_TELEMETRY.tex	ASPLOS camera-ready LaTeX (acmart sigplan)
paper/EESM_GPU_TELEMETRY.md	Markdown manuscript
paper/reference_tables.json	Machine-readable Table benchmarks
manifests/h100_telemetry.json	H100 cluster + kernel-class lock
src/python/eesm/pipeline.py	EpigeneticPipeline
src/python/eesm/interventions.py	SCM do(δ) operators
src/python/eesm/markov.py	Bregman + Viterbi S_markov
src/python/eesm/stream.py	T_stream partial order
src/python/eesm/embedding.py	O_θ + B_bounded
src/python/eesm/covariance.py	κ(Σ_k) detector
tools/fetch_trace_corpus.py	Trace corpus ingest / demo
tools/verify_audit.py	Interventions + ablations + ledger
VALIDATION.md	Platform smoke-test notes

---

Citation

@inproceedings{eesm_gpu_telemetry_2026,
  title={Epigenetic Execution-State Modeling for Causal Invariance in GPU Performance Telemetry},
  author={Anonymous Authors},
  year={2026},
  note={https://github.com/FractiAI/eesm-gpu-telemetry}
}