engine: \begin and \end are not the gullet's to expand - #182
Merged
Conversation
#179 says it runs "the substituted material" through the gullet; it ran the WHOLE maths source. The rest of that source is the author's maths, and the engine's text-mode stand-ins won there: \def\cdot{\char183\relax} (latex.go) \def\begin#1{\gotex@checkenv{#1}\csname #1\endcsname} so every \cdot became \char — 935 dropped formulas over 45 papers — and every \begin{bmatrix} became \bmatrix, 676 over 42. Worse than the drops: with \begin expanded, \gotex@checkenv opened the environment and the ENGINE typeset the matrix as text. One paper came out with 411251 glyphs on seven pages, 58000 a page, where it has 69284. Now only the body just substituted is flattened, which is what the seam was for: a class's \the@inst is \number\c@inst and that still resolves. \begin and \end are marked \noexpand inside the body too, for a macro that wraps an environment. Measured over 200 arXiv papers and 200 beamer talks, against main: formulas the maths layer refuses 4312 -> 2715 arXiv page error (157 renders) 624 -> 634, exacts 17 -> 19 the flooded paper 411251 -> 69284 glyphs The page error moves the wrong way by ten because the flood was flattering it. Against the state before #179 the pair now stands at 651 -> 634 with 14 -> 19 exact, for 212 more dropped formulas — a trade the exact matches and the junk removal earn.
tannevaled
force-pushed
the
math-protect-begin
branch
from
September 1, 2026 08:22
795f924 to
5b3a1ac
Compare
tannevaled
added a commit
that referenced
this pull request
Sep 1, 2026
The engine spells some characters it has no glyph command for as \char:
\def\cdot{\char183\relax} \def\bullet{\char8226\relax} (latex.go)
\protected\def\_{\char95\relax} (classkernel.go)
In a formula those names belong to the maths layer, which renders ⋅ and ∙ itself. A
substituted body is flattened through the gullet (#179, #182), and flattening turned
them into \char — a primitive the layer refuses, costing the whole formula. A paper's
own \expect{}{\cdot} lost every occurrence.
A body that begins with \char is what makes a stand-in recognisable, without a list
to keep in step with the kernel; those names are marked \noexpand for the flattening,
beside \begin and \end.
Measured over 200 arXiv papers and 200 beamer talks: the formulas the maths layer
refuses fall 2715 -> 2635, arXiv gains 860 glyphs over 8 documents and one more exact
page match (20 of 157), the page error 634 -> 633. beamer does not move.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#179 flattens a substituted macro body through the gullet, which is what reaches a class's
\numberand\expandafter. But\beginand\endare macros here:so expanding them turned
\begin{bmatrix}into\bmatrix— a control sequence the maths layer has never heard of, where the environment it wrote is one it renders. A paper's ownlost every formula that used it, and over the corpus
bmatrix,pmatrix,cases,aligned,splitandarrayaccounted for 676 fresh drops in 42 papers.Worse than the drops: with
\beginexpanded,\gotex@checkenvopened the environment and the engine typeset the matrix as text. One paper came out with 411251 glyphs on seven pages — 58 000 a page — where it has 69284.Both are the same fix: the two tokens are marked
\noexpandbefore the flattening.Measured — 200 arXiv papers, 200 beamer talks
The page error moves the wrong way by four because the flood was flattering it; against 651 before #179, the pair is still well ahead. beamer moves one page.
Test
mathprotectbegin_test.go: a macro whose body is\begin{bmatrix}…\end{bmatrix}, one withcases, one witharray— each must leave the maths layer with nothing to drop. All three fail onmainwithmap[\bmatrix:1],map[\cases:1],map[\array:1].