-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathobserver_paper.html
More file actions
2029 lines (1849 loc) · 90.9 KB
/
Copy pathobserver_paper.html
File metadata and controls
2029 lines (1849 loc) · 90.9 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>observer: Runtime Instrumentation for Trajectory Mapping in Language Models</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link href="https://fonts.googleapis.com/css2?family=DM+Mono:ital,wght@0,300;0,400;0,500;1,300&family=Fraunces:ital,opsz,wght@0,9..144,300;0,9..144,400;0,9..144,600;1,9..144,300;1,9..144,400&family=DM+Sans:ital,wght@0,300;0,400;0,500;1,300&display=swap" rel="stylesheet">
<style>
:root {
--bg: #0d0f0e;
--surface: #131614;
--surface2: #1a1d1b;
--border: #252925;
--border2: #2f342f;
--text: #d4d9d4;
--text-dim: #7a8a7a;
--text-dimmer: #4a574a;
--accent: #6ee87a;
--accent2: #3adc4a;
--accent-dim: #1a3a1e;
--warn: #e8c46e;
--warn-dim: #3a2e1a;
--code-bg: #0a0c0a;
--red: #e86e6e;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
html {
font-size: 16px;
background: var(--bg);
color: var(--text);
scroll-behavior: smooth;
}
body {
font-family: 'DM Sans', sans-serif;
font-weight: 300;
line-height: 1.75;
max-width: 860px;
margin: 0 auto;
padding: 0 2rem 8rem;
}
/* ─── HEADER ─── */
.paper-header {
padding: 5rem 0 3rem;
border-bottom: 1px solid var(--border);
margin-bottom: 3rem;
animation: fadeUp 0.8s ease both;
}
.label-row {
display: flex;
align-items: center;
gap: 1rem;
margin-bottom: 1.5rem;
}
.label {
font-family: 'DM Mono', monospace;
font-size: 0.7rem;
letter-spacing: 0.15em;
text-transform: uppercase;
color: var(--accent);
background: var(--accent-dim);
padding: 0.25em 0.75em;
border-radius: 2px;
}
.label-plain {
font-family: 'DM Mono', monospace;
font-size: 0.7rem;
letter-spacing: 0.15em;
text-transform: uppercase;
color: var(--text-dim);
}
h1 {
font-family: 'Fraunces', serif;
font-size: clamp(2rem, 5vw, 3.25rem);
font-weight: 300;
line-height: 1.15;
letter-spacing: -0.02em;
color: #e8ede8;
margin-bottom: 1.25rem;
}
h1 em {
font-style: italic;
color: var(--accent);
}
.subtitle {
font-family: 'DM Sans', sans-serif;
font-size: 1.05rem;
color: var(--text-dim);
font-weight: 300;
max-width: 640px;
line-height: 1.65;
}
.meta-row {
display: flex;
flex-wrap: wrap;
gap: 2rem;
margin-top: 2rem;
padding-top: 2rem;
border-top: 1px solid var(--border);
}
.meta-item {
font-family: 'DM Mono', monospace;
font-size: 0.75rem;
color: var(--text-dimmer);
}
.meta-item span {
color: var(--text-dim);
display: block;
margin-top: 0.2em;
}
/* ─── SECTIONS ─── */
section {
margin-bottom: 3.5rem;
animation: fadeUp 0.6s ease both;
}
h2 {
font-family: 'Fraunces', serif;
font-size: 1.5rem;
font-weight: 400;
color: #e8ede8;
letter-spacing: -0.01em;
margin-bottom: 1.25rem;
padding-bottom: 0.5rem;
border-bottom: 1px solid var(--border);
display: flex;
align-items: baseline;
gap: 0.75rem;
}
h2 .sec-num {
font-family: 'DM Mono', monospace;
font-size: 0.7rem;
color: var(--accent);
letter-spacing: 0.1em;
}
h3 {
font-family: 'DM Sans', sans-serif;
font-size: 0.9rem;
font-weight: 500;
color: var(--accent);
letter-spacing: 0.08em;
text-transform: uppercase;
margin: 2rem 0 0.75rem;
}
p {
font-size: 0.975rem;
color: var(--text);
margin-bottom: 1rem;
font-weight: 300;
}
strong {
font-weight: 500;
color: #e8ede8;
}
em {
font-style: italic;
color: var(--text-dim);
}
/* ─── ABSTRACT BOX ─── */
.abstract {
background: var(--surface);
border: 1px solid var(--border);
border-left: 3px solid var(--accent);
padding: 1.75rem 2rem;
border-radius: 4px;
margin-bottom: 3.5rem;
animation: fadeUp 0.7s ease both;
}
.abstract-label {
font-family: 'DM Mono', monospace;
font-size: 0.65rem;
letter-spacing: 0.2em;
text-transform: uppercase;
color: var(--accent);
margin-bottom: 0.75rem;
}
.abstract p {
font-size: 0.95rem;
line-height: 1.8;
color: var(--text-dim);
margin-bottom: 0;
}
/* ─── CODE ─── */
pre {
background: var(--code-bg);
border: 1px solid var(--border);
border-radius: 4px;
padding: 1.5rem;
overflow-x: auto;
margin: 1.25rem 0;
position: relative;
}
pre .code-label {
position: absolute;
top: 0.6rem;
right: 0.75rem;
font-family: 'DM Mono', monospace;
font-size: 0.6rem;
letter-spacing: 0.12em;
text-transform: uppercase;
color: var(--text-dimmer);
}
code {
font-family: 'DM Mono', monospace;
font-size: 0.8rem;
line-height: 1.7;
color: #9ecf9e;
}
p code, li code {
background: var(--surface2);
border: 1px solid var(--border);
padding: 0.1em 0.4em;
border-radius: 2px;
font-size: 0.8rem;
color: var(--accent);
}
.comment { color: var(--text-dimmer); font-style: italic; }
.kw { color: #7ec8e3; }
.fn { color: #e8c46e; }
.str { color: #b5e8b5; }
.num { color: #e8a06e; }
/* ─── CALLOUT ─── */
.callout {
background: var(--warn-dim);
border: 1px solid var(--warn);
border-radius: 4px;
padding: 1.25rem 1.5rem;
margin: 1.5rem 0;
}
.callout p {
color: var(--warn);
font-size: 0.9rem;
margin: 0;
}
.callout-green {
background: var(--accent-dim);
border-color: var(--accent);
}
.callout-green p {
color: var(--accent);
}
/* ─── FORMULA ─── */
.formula {
background: var(--code-bg);
border: 1px solid var(--border);
border-radius: 4px;
padding: 1.25rem 2rem;
margin: 1.25rem 0;
text-align: center;
font-family: 'DM Mono', monospace;
font-size: 0.85rem;
color: var(--accent);
line-height: 2;
overflow-x: auto;
}
/* ─── TABLE ─── */
.table-wrap {
overflow-x: auto;
margin: 1.5rem 0;
}
table {
width: 100%;
border-collapse: collapse;
font-size: 0.85rem;
}
th {
font-family: 'DM Mono', monospace;
font-size: 0.65rem;
letter-spacing: 0.12em;
text-transform: uppercase;
color: var(--accent);
border-bottom: 1px solid var(--border2);
padding: 0.6rem 1rem;
text-align: left;
font-weight: 400;
}
td {
padding: 0.6rem 1rem;
border-bottom: 1px solid var(--border);
color: var(--text-dim);
font-size: 0.875rem;
vertical-align: top;
}
tr:last-child td { border-bottom: none; }
td strong { color: var(--text); }
td code { font-size: 0.75rem; }
/* ─── FIGURE / DIAGRAM ─── */
.diagram {
background: var(--code-bg);
border: 1px solid var(--border);
border-radius: 4px;
padding: 2rem;
margin: 1.5rem 0;
font-family: 'DM Mono', monospace;
font-size: 0.75rem;
color: var(--text-dim);
line-height: 1.8;
overflow-x: auto;
}
.diagram .d-accent { color: var(--accent); }
.diagram .d-warn { color: var(--warn); }
.diagram .d-dim { color: var(--text-dimmer); }
.fig-caption {
font-family: 'DM Mono', monospace;
font-size: 0.7rem;
color: var(--text-dimmer);
text-align: center;
margin-top: 0.5rem;
letter-spacing: 0.05em;
}
/* ─── REGIME GRID ─── */
.regime-grid {
display: grid;
grid-template-columns: repeat(2, 1fr);
gap: 0.75rem;
margin: 1.25rem 0;
}
.regime-card {
background: var(--surface);
border: 1px solid var(--border);
border-radius: 4px;
padding: 1rem 1.25rem;
}
.regime-card .rc-label {
font-family: 'DM Mono', monospace;
font-size: 0.7rem;
letter-spacing: 0.12em;
margin-bottom: 0.4rem;
}
.rc-elastic { color: var(--accent); }
.rc-partial { color: #6ebde8; }
.rc-plastic { color: var(--warn); }
.rc-divergent { color: var(--red); }
.regime-card p {
font-size: 0.82rem;
color: var(--text-dim);
margin: 0;
}
/* ─── COMPONENT STACK ─── */
.stack {
display: flex;
flex-direction: column;
gap: 0.5rem;
margin: 1.25rem 0;
}
.stack-item {
display: flex;
align-items: stretch;
gap: 0;
border: 1px solid var(--border);
border-radius: 4px;
overflow: hidden;
}
.stack-tag {
font-family: 'DM Mono', monospace;
font-size: 0.65rem;
letter-spacing: 0.08em;
padding: 0.75rem 0.9rem;
background: var(--surface2);
color: var(--accent);
border-right: 1px solid var(--border);
min-width: 60px;
display: flex;
align-items: center;
justify-content: center;
text-transform: uppercase;
writing-mode: horizontal-tb;
white-space: nowrap;
}
.stack-content {
padding: 0.75rem 1.25rem;
flex: 1;
}
.stack-content strong {
font-size: 0.85rem;
display: block;
margin-bottom: 0.2rem;
}
.stack-content p {
font-size: 0.8rem;
color: var(--text-dim);
margin: 0;
}
/* ─── FOOTNOTES ─── */
.footnotes {
border-top: 1px solid var(--border);
margin-top: 4rem;
padding-top: 1.5rem;
font-family: 'DM Mono', monospace;
font-size: 0.72rem;
color: var(--text-dimmer);
line-height: 1.7;
}
.footnotes a {
color: var(--accent);
text-decoration: none;
}
/* ─── ANIMATIONS ─── */
@keyframes fadeUp {
from { opacity: 0; transform: translateY(16px); }
to { opacity: 1; transform: translateY(0); }
}
section:nth-child(1) { animation-delay: 0.05s; }
section:nth-child(2) { animation-delay: 0.1s; }
section:nth-child(3) { animation-delay: 0.15s; }
section:nth-child(4) { animation-delay: 0.2s; }
section:nth-child(5) { animation-delay: 0.25s; }
/* ─── RESPONSIVE ─── */
@media (max-width: 600px) {
body { padding: 0 1rem 5rem; }
.regime-grid { grid-template-columns: 1fr; }
.meta-row { gap: 1rem; }
}
</style>
</head>
<body>
<header class="paper-header">
<div class="label-row">
<span class="label">Preprint · v2 + foundation correction</span>
<span class="label-plain">v1 February 2026 · v2 April 2026 · correction July 2026</span>
</div>
<h1><em>observer</em>: Runtime Instrumentation for<br>Trajectory Mapping in Language Models</h1>
<p class="subtitle">
A runtime instrumentation stack for measuring perturbation dynamics, characterizing
branchpoint geometry, and conducting falsifiable closed-loop control experiments
during autoregressive generation — without modifying model weights.
</p>
<div class="meta-row">
<div class="meta-item">AUTHOR<span>Josh Malone</span></div>
<div class="meta-item">REPOSITORY<span>github.com/aeon0199/observer</span></div>
<div class="meta-item">LICENSE<span>MIT</span></div>
</div>
</header>
<div class="callout callout-green" style="margin-bottom: 2.5rem;">
<p>
<strong>Foundation rebuild status (2026-07-19).</strong> A code-and-claims
audit rebuilt Observer around <strong>Observe → Perturb → Compare →
Prove</strong>, with Act/controller work downstream and paused. The active
implementation now distinguishes consumed tokens from predicted next
tokens, records authoritative loaded-model provenance, creates independent
same-context one-step branchpoint forks, runs matched clean/perturbed
recovery branches, and validates diagnostics against simple baselines and
known synthetic trajectories.
</p>
<p>
This audit also reopens two v2 interpretations. First, the F31 AUROCs came
from continued controller shadow/active trajectories. After the first token
difference, later rows had different contexts, so their labels measured
shifted/cascading divergence rather than independent local flippability.
Second, the historical BASE → PERTURB → REASK protocol changed context and
continued only the perturbed branch; its perturbation effects remain
observations, but its recovery ratios and regime labels are provisional.
The historical numbers below are retained as an audit record, not erased.
The current contract is documented in
<code>docs/foundations/OBSERVER_FOUNDATIONS.md</code>.
</p>
<p>
The repaired protocols were then calibrated in 42 Qwen3-1.7B runs across
three seeds and two prompts. At relative additive magnitude 0.30, unique
whole-run prompt-slice AUROC reached 0.831 on sourdough and 0.886 on water
cycle. Matched hysteresis showed near-zero and order-one endpoint
propagation at the same magnitude on different seeds. These new findings
are F32/F33 below; they do not resume the controller.
</p>
<p>
A subsequent 280-run Q2 sweep captured paired deltas at every downstream
block and final RMSNorm. It found that raw L2 amplification, normalized
structural disturbance, and output disruption are not interchangeable.
Final RMSNorm erases radial final-layer scaling but preserves directional
additive change. F34 records the scoped result. A subsequent 60-run Q3
suite continued verified clean and perturbed branches across ten prompts.
It observed improve, degrade, and tie outcomes under frozen task rubrics,
but a clean pre-flip predictor reached only 0.571 mean whole-prompt-held-out
AUROC, below the prespecified 0.70 gate. F35 records that measured negative
result. A preregistered 90-run M3R-2 replication then used 15 entirely new
prompts, higher-resolution frozen rubrics, and a compact regularized
predictor. It reached 0.596 mean AUROC across 13 valid prompt holdouts,
still below 0.70, and the earlier procedural direction did not replicate.
F36 records the strengthened negative result; controller work remains
paused.
</p>
</div>
<div class="callout callout-green" style="margin-bottom: 2.5rem;">
<p>
<strong>v2 update (2026-04-19).</strong> The first version of this paper presented
closed-loop stability control as the central contribution. Subsequent experiments
falsified the original controller thesis on Qwen3-1.7B: scaling interventions at
the final layer have zero effect (absorbed by RMSNorm), additive interventions are
either silent no-ops or over-actuating depending on magnitude, and acting earlier
in the residual stream cascades destructively. Most fundamentally, the
divergence signal that drives the controller was found to measure token-level prose
surprise — word-starts, semantic transitions, structural boundaries — rather
than dynamical instability in any control-theoretic sense.
</p>
<p>
What survives is a repairable instrument and a set of observed perturbation
effects. The July foundation audit replaces the old branchpoint and
hysteresis interpretations with matched protocols. The historical
0.82–0.86 AUROCs are no longer presented as evidence that local flippability
is predictable; F31 and Q1 are reopened for calibration with independent
one-step counterfactuals.
</p>
<p>
Sections rewritten in v2: §6 (spectral diagnostics, methodology corrected), §9
(controller empirical evaluation, replaced with falsification arc), §10
(experimental results, replaced with the Qwen3-1.7B controller arc), §11 (new —
branchpoint hijacking), §12 (new — mapping program), §14 (limitations updated),
§15 (conclusion rewritten). The July 2026 correction subsequently rewrote
§3 architecture, §7 hysteresis, §11 F31, §12 status, §13 reproducibility,
§14 limitations, and §15 conclusion around the repaired foundation.
</p>
</div>
<div class="abstract">
<div class="abstract-label">Abstract</div>
<p>
We present <strong>observer</strong>, an open-source runtime stack for studying
perturbation dynamics in autoregressive language models. Its active foundation is
Observe → Perturb → Compare → Prove: canonical token-time telemetry,
independent local counterfactual forks, persistent propagation experiments,
matched recovery, and provenance-rich artifacts. The historical
<code>divergence</code> field is a compatibility alias for a descriptive
local VAR(1) prediction error. Simple hidden-velocity, hidden-acceleration,
logit-entropy, and top-1-margin baselines are emitted beside the advanced
probes.
</p>
<p>
We use this instrument to falsify our own initial closed-loop stability control
hypothesis on Qwen3-1.7B and report what the experiments actually showed: the
divergence signal correlates with token-level prose surprise (word-starts,
structural boundaries, semantic transitions) rather than dynamical instability in
a control-theoretic sense. Closed-loop control over this signal is not effective
on the tested model. We then report an observed effect from the same
apparatus: additive interventions can produce token and trajectory changes.
However, a later audit found that the reported 0.82 and 0.86 AUROCs used
shifted/cascading labels from continued controller trajectories and therefore
do not establish predictable local branchpoints. We preserve that negative
methodology lesson, describe the repaired matched protocols, and reopen the
mapping questions with explicit stop conditions and controller-return criteria.
</p>
</div>
<section>
<h2><span class="sec-num">§1</span> Motivation</h2>
<p>
The dominant paradigm in mechanistic interpretability — sparse autoencoders, circuit discovery, logit lens analysis —
answers the question <em>"what does this model compute?"</em> It is fundamentally a post-hoc analytical approach.
The field has produced significant understanding of model internals, but has largely deferred a different class of question:
</p>
<p>
<strong>Can we detect when generation is destabilizing, in real time, and do something about it?</strong>
</p>
<p>
This is the question observer is built to answer. It is closer in spirit to control engineering than to interpretability
research: rather than analyzing a system's internal structure, we treat the model as a dynamical system
and ask whether we can build a feedback loop around it.
</p>
<p>
The practical stakes are not abstract. High-stakes deployments of language models — in agentic settings,
long-horizon tasks, adversarial environments — require some answer to the question of whether generation
has gone off course and whether that course can be corrected. The current state of the art is largely
output-level heuristics: does the text look wrong? Observer proposes that the answer should be
visible in the hidden trajectory before it surfaces in the output, and that a runtime controller
can act on that signal.
</p>
<div class="callout callout-green">
<p>
<strong>Scope caveat (v1, retained for context):</strong> Observer was framed as a
research instrument. The original paper hedged that the divergence signal measured
trajectory instability and that empirical validation of downstream correlates was
"the necessary next step."
</p>
<p>
<strong>Update (v2):</strong> we performed that validation. The divergence signal
does not measure dynamical instability in the control-theoretic sense it was
framed as measuring; it measures token-level prose surprise. See §10 for the
falsification arc and §11 for the observed perturbation effects and
reopened local-branchpoint question that emerged from the same apparatus.
</p>
</div>
</section>
<section>
<h2><span class="sec-num">§2</span> Related Work</h2>
<p>
Observer occupies a space adjacent to several lines of existing work, without directly duplicating any of them.
</p>
<h3>Intervention Tooling</h3>
<p>
<strong>TransformerLens</strong> (Nanda, 2022) provides the dominant toolkit for mechanistic interpretability
research: model loading, hook-based activation capture and modification, and a large community of research
built on its abstractions. It is an exploration tool — excellent for research notebooks and circuit analysis,
not designed around systematic experimental protocols or recovery measurement.
</p>
<p>
<strong>pyvene</strong> (Wu et al., 2024) formalizes interventions as first-class serializable primitives,
enabling composable intervention specifications across locations, granularity, and sequence position.
It is an execution library: it provides the mechanics of intervention without opinions about experimental design,
hysteresis, or recovery.
</p>
<p>
<strong>nnsight</strong> provides a Pythonic interface for local and remote
model execution, including NDIF infrastructure. Observer's current optional
nnsight adapter supports local execution only. Remote execution is
intentionally rejected until Observer has a trace-based backend that can
preserve its event and intervention contracts honestly.
</p>
<h3>Representation Engineering and Steering</h3>
<p>
The Representation Engineering paper (Zou et al., 2023) demonstrated that model behavioral tendencies
can be read from and written to activation space via linear probes and steering vectors. The Inference-Time
Intervention paper (Li et al., 2023) applied shifted activations at inference time, improving TruthfulQA
performance from 32.5% to 65.1%. Neither line of work focused on recovery dynamics or closed-loop feedback.
</p>
<h3>LLM Stability</h3>
<p>
Recent work on LLM output consistency (Raj et al., 2023; Huang et al., 2023) characterizes stability
at the output level — how often does the same model produce the same answer across runs?
Observer operates at a different layer: activation-level perturbation dynamics within a single generation,
not output-level consistency across generations.
</p>
<h3>Recent Developments (2025–2026)</h3>
<p>
<strong>LinEAS</strong> (Rodriguez et al., NeurIPS 2025; arXiv:2503.10679) trains activation steering end-to-end
with a global distributional loss, showing that locally tuned maps produce unintended downstream shifts when
applied out-of-sample. Observer's adaptive controller is designed to detect and respond to such downstream
cascades in real time.
</p>
<p>
<strong>FASB</strong> (Cheng et al., 2025; arXiv:2508.17621) dynamically determines intervention necessity and
strength by tracking internal states during generation, with a backtracking mechanism to correct deviated tokens.
Observer shares the adaptive framing but adds deterministic branchpointing and explicit recovery measurement,
quantifying whether the trajectory recovered or remained shifted after intervention ended.
</p>
<p>
Grant et al. (2025; arXiv:2511.04638) provide a theoretical treatment of how causal interventions can push
representations off the model's natural manifold, distinguishing benign null-space divergences from pernicious
ones that activate dormant pathways. Observer's PLASTIC and DIVERGENT regime classifications can be interpreted
through this taxonomy, offering empirical runtime signatures for divergence types their framework characterizes
theoretically.
</p>
<p>
<strong>HARP</strong> (Hu et al., 2025; arXiv:2509.11536) decomposes hidden state space into semantic and
reasoning subspaces via SVD of the unembedding layer, achieving AUROC 92.8% on TriviaQA hallucination detection.
Observer's windowed SVD probe tracks effective rank dynamically within a generation rather than using static
subspace decomposition for classification, a complementary signal.
</p>
<p>
<strong>HALT</strong> (Shapiro, Taneja, and Goel, Feb 2026; arXiv:2602.02888) treats token log-probability
sequences as a time series for lightweight hallucination detection without requiring internal model access.
Observer's VAR(1) predictor applies a related time-series framing to hidden state trajectories, a white-box
signal that feeds an active intervention loop rather than a post-hoc detector.
</p>
<h3>The Gap</h3>
<p>
Observer's active contribution is experimental discipline around runtime
intervention: canonical causal events, deterministic same-context local
forks, persistent propagation experiments, equal matched-recovery branches,
and provenance that makes matching claims checkable. Existing tools execute
interventions; Observer separates what changed immediately, what propagated,
what persisted, and what recovered. Closed-loop action remains a downstream,
paused research direction.
</p>
</section>
<section>
<h2><span class="sec-num">§3</span> Architecture Overview</h2>
<p>
Observer's active architecture follows four evidence stages. Historical
versioned directories remain in the repository for reproduction, but they
are not the current protocol contract.
</p>
<div class="stack">
<div class="stack-item">
<div class="stack-tag">01</div>
<div class="stack-content">
<strong>Observe — Canonical Runtime Events</strong>
<p>Record the consumed token, pre/post-intervention hidden state, logits,
predicted next token, simple baselines, and advanced token-time
diagnostics without assigning unsupported semantic meaning.</p>
</div>
</div>
<div class="stack-item">
<div class="stack-tag">02</div>
<div class="stack-content">
<strong>Perturb — Local or Persistent Counterfactuals</strong>
<p>Use independent one-step same-context forks for local sensitivity, or
continued matched branches for propagation. Do not mix those labels.</p>
</div>
</div>
<div class="stack-item">
<div class="stack-tag">03</div>
<div class="stack-content">
<strong>Compare — Sensitivity, Propagation, Persistence, Recovery</strong>
<p>Measure hidden and logit distances, token flips, exposure endpoints,
and equal intervention-free matched recovery continuations.</p>
</div>
</div>
<div class="stack-item">
<div class="stack-tag">04</div>
<div class="stack-content">
<strong>Prove — Provenance and Falsification</strong>
<p>Bind every result to loaded-model identity, exact config hash,
context fingerprints, protocol-validity fields, structured events,
zero controls, and tests. Act/controller work is downstream.</p>
</div>
</div>
</div>
<div class="diagram">
<pre style="background:none;border:none;padding:0;margin:0;font-size:0.75rem;">
<span class="d-dim">PROMPT</span>
│
▼
<span class="d-accent">[ SeedCache: build_seed_cache() ]</span>
│ past_key_values snapshot
│ next_token_logits
│ seed_hidden @ intervention_layer
│
├──────────────────────────┐
▼ ▼
<span class="d-accent">[ CLEAN one-step fork ]</span> <span class="d-warn">[ PERTURBED one-step fork ]</span>
same consumed token same consumed token
same sampling RNG intervention active
│ │
└──────────┬───────────────┘
▼
<span class="d-accent">[ Local causal comparison ]</span>
hidden cosine / relative L2
KL / JS on logits
argmax and sampled-token flip
discard perturbed fork
advance clean trajectory only</pre>
</div>
<div class="fig-caption">Figure 1. Local branchpoint data flow. Every row is an independent same-context one-step counterfactual.</div>
</section>
<section>
<h2><span class="sec-num">§4</span> SeedCache: Deterministic Branchpointing</h2>
<p>
The central design problem in intervention experiments is confounding. A naive implementation runs
the baseline and intervention branches from separate forward passes over the same prompt.
This introduces at minimum: different random number generator states at the point of token sampling
(even under greedy decoding, CUDA operations can have ordering nondeterminism), and potentially
different attention mask states depending on the batching implementation.
</p>
<p>
The SeedCache resolves this by running the prompt <strong>exactly once</strong>, then cloning
the resulting model state for both branches:
</p>
<pre><code><span class="comment"># Run prompt once, snapshot pre-generation state</span>
<span class="kw">def</span> <span class="fn">build_seed_cache</span>(model, tokenizer, device, prompt, layer) -> SeedCache:
hook = _HiddenCaptureHook()
handle = layers[layer].register_forward_hook(hook)
<span class="kw">with</span> torch.no_grad():
outputs = model(input_ids, use_cache=<span class="kw">True</span>, return_dict=<span class="kw">True</span>)
handle.remove()
<span class="kw">return</span> SeedCache(
past_key_values = outputs.past_key_values, <span class="comment"># full KV cache</span>
next_token_logits = outputs.logits[:,-1,:], <span class="comment"># first token dist</span>
seed_hidden = hook.captured, <span class="comment"># hidden @ layer</span>
fingerprint = compute_cache_fingerprint(...) <span class="comment"># checksum</span>
)
<span class="comment"># Both branches start from identical state</span>
baseline_cache = seed_cache.clone()
intervention_cache = seed_cache.clone()
<span class="comment"># SeedCache.clone() deep-copies past_key_values via clone_past_key_values()</span>
<span class="comment"># handles DynamicCache, legacy tuple-of-tuples, and generic objects</span></code><span class="code-label">cache.py</span></pre>
<p>
The fingerprint hashes the first-layer key-cache shape, dtype, and a
deterministic value sample. Experiments use it with cache sequence length
to verify both forks share a common origin. This is the kind of
rigor that most published intervention papers treat as an implementation detail but actually matters
for result validity.
</p>
<div class="callout">
<p>
<strong>Why this matters:</strong> Without a shared branchpoint, local
sensitivity measurements conflate intervention effects with different
initial state. Recovery additionally requires two aligned continuations,
as specified in §7.
</p>
</div>
</section>
<section>
<h2><span class="sec-num">§5</span> The Local Prediction-Error Signal</h2>
<p>
The signal historically called <code>divergence</code> is a
<strong>per-token held-out local prediction error</strong> from a VAR(1)
model fit on a sliding window of projected hidden states. Canonical
diagnostics now expose <code>local_prediction_error</code> and retain
<code>divergence</code> as a compatibility alias.
</p>
<h3>Projection</h3>
<p>
The hidden state <strong>h</strong><sub>t</sub> ∈ ℝ<sup>D</sup> (where D is the model's hidden dimension,
typically 4096–8192) is projected to a fixed low-dimensional space via a deterministic Rademacher matrix:
</p>
<div class="formula">
z<sub>t</sub> = h<sub>t</sub> · P, P ∈ {±1/√k}<sup>D×k</sup>, k = 64
<br>
P is fixed for the run lifetime (seeded, deterministic)
</div>
<p>
The Rademacher projection preserves inner products in expectation (Johnson-Lindenstrauss),
reduces the regression problem from D-dimensional to k-dimensional (k=64),
and is computed once per hidden dimension via a seeded RNG — making it reproducible
across runs and comparable across model families with different hidden sizes.
</p>
<h3>VAR(1) Dynamics</h3>
<p>
A first-order vector autoregressive model is fit on the sliding window
W = {z<sub>t-n</sub>, ..., z<sub>t-1</sub>} via ridge regression:
</p>
<div class="formula">
z<sub>t</sub> ≈ z<sub>t-1</sub> · A, A ∈ ℝ<sup>k×k</sup>
<br>
(X<sup>T</sup>X + λI) A = X<sup>T</sup>Y, λ = 0.01
</div>
<p>
Critically, the matrix A is fit on the window <em>excluding the newest state</em> z<sub>t</sub>.
The prediction ẑ<sub>t</sub> = z<sub>t-1</sub> · A is then compared to the actual observed z<sub>t</sub>.
This is a held-out evaluation: the model is never trained on the transition it is asked to predict.
This matters because in-sample VAR(1) error on a short window would collapse toward zero regardless of
actual trajectory instability.
</p>
<h3>Divergence Score</h3>
<p>
The per-token scalar divergence combines normalized L2 error and cosine distance with a
symmetric denominator to avoid blow-ups when projected norms are near zero:
</p>
<div class="formula">
L2<sub>norm</sub> = ||ẑ<sub>t</sub> − z<sub>t</sub>|| / (0.5 · (||ẑ<sub>t</sub>|| + ||z<sub>t</sub>||) + ε)
<br>
cos<sub>dist</sub> = 1 − (ẑ<sub>t</sub> · z<sub>t</sub>) / (||ẑ<sub>t</sub>|| · ||z<sub>t</sub>|| + ε)
<br>
divergence = 0.7 · L2<sub>norm</sub> + 0.3 · cos<sub>dist</sub>
</div>
<p>
When the hidden trajectory is locally predictable, the VAR(1) fit is good and divergence is low.
When generation dynamics shift — through perturbation, distributional shift in the prompt context,
or internal instability — the held-out prediction error increases. The signal is cheap:
one matrix multiply per token in 64-dimensional space.
</p>
<pre><code><span class="kw">def</span> <span class="fn">step</span>(self, hidden: torch.Tensor) -> float:
z = self._project(hidden) <span class="comment"># (D,) → (64,)</span>
self._window.add(z) <span class="comment"># FIFO buffer, maxlen=8</span>
<span class="kw">if</span> <span class="fn">len</span>(self._window) < <span class="num">3</span>:
<span class="kw">return</span> <span class="num">0.0</span>
states = self._window.matrix() <span class="comment"># (T, 64)</span>
train = states[:-<span class="num">1</span>, :] <span class="comment"># exclude newest</span>
A = _fit_var1_ridge(train) <span class="comment"># fit on T-1 transitions</span>
pred = states[-<span class="num">2</span>, :] @ A <span class="comment"># predict from t-1</span>
actual = states[-<span class="num">1</span>, :] <span class="comment"># held-out: actual t</span>
<span class="kw">return</span> _divergence(pred, actual)[<span class="str">"combined"</span>]</code><span class="code-label">predictor.py</span></pre>
</section>
<section>
<h2><span class="sec-num">§6</span> Supplementary Diagnostics</h2>
<p>
The divergence signal is the primary input to the controller, but the V1.5 observability
runner and the adaptive controller also compute three supplementary diagnostics that provide
corroborating signal and richer telemetry for offline analysis.
</p>
<h3>Spectral Diagnostics</h3>
<div class="callout callout-green">
<p>
<strong>v2 correction.</strong> The original spectral module FFT'd the flattened
hidden-state vector along the feature-index axis and reported entropy, flatness,
centroid, and band fractions over that spectrum. The v1 paper acknowledged that
"the feature index is not a temporal axis" but defended the metrics as a stable
characterization of activation energy distribution. This defense does not survive
a permutation test: neuron ordering in transformer hidden states is arbitrary
(a function of weight initialization, not semantics), and any neuron-axis FFT
summary is a function of that arbitrary ordering. Permuting neurons changes every
reported metric; the underlying activation is unchanged.
</p>
<p>
The v2 implementation rewrites this module as a token-time spectral probe:
hidden states are accumulated into a sliding window of shape <code>[T, D]</code>
and the FFT is taken along the time axis (dim=0). Per-frequency power is then
averaged across the D dimensions, producing a scalar trajectory spectrum. This
captures real structure — slow drift vs. high-frequency oscillation in
activation patterns across generation steps — and is invariant to neuron
permutation. A built-in self-test reports a non-zero permutation-change ratio
whenever the window has at least 8 tokens, confirming the time axis is in fact
what's being analyzed.
</p>
</div>
<p>
The corrected metrics, computed on the time-axis trajectory spectrum, are:
</p>
<div class="table-wrap">
<table>
<thead>
<tr><th>Metric</th><th>Description</th></tr>
</thead>
<tbody>
<tr><td><code>spectral_entropy</code></td><td>Normalized Shannon entropy of the time-axis power spectrum. High = energy spread across slow and fast trajectory frequencies.</td></tr>
<tr><td><code>spectral_flatness</code></td><td>Geometric mean / arithmetic mean of power. Approaches 1.0 for white-noise trajectories, 0.0 for tonally pure ones.</td></tr>
<tr><td><code>centroid</code></td><td>Normalized frequency centroid ∈ [0,1]. High centroid = trajectory dominated by step-to-step oscillation rather than slow drift.</td></tr>
<tr><td><code>high_frac</code></td><td>Fraction of power in the upper 20% of trajectory frequencies.</td></tr>
<tr><td><code>rolloff_85</code></td><td>Normalized frequency below which 85% of cumulative power falls.</td></tr>
<tr><td><code>permutation_change</code></td><td><em>(new in v2)</em> Self-test ratio comparing the spectrum of the actual trajectory to the spectrum of a randomly time-permuted version of the same window. Should be > 0 — confirms time-axis behavior. If a future regression makes this near zero we know the module has reverted to neuron-axis behavior.</td></tr>
</tbody>
</table>
</div>