Skip to content

[S4] Quantize, and emit the error bound into the generated table - #50

Merged
swgiacomelli merged 18 commits into
epic/36-surfaces-bakefrom
cursor/issue-40-s4-quantize-03fb
Aug 25, 2026
Merged

swgiacomelli merged 18 commits into
epic/36-surfaces-bakefrom
cursor/issue-40-s4-quantize-03fb

Conversation

@swgiacomelli

@swgiacomelli swgiacomelli commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Closes #40.

Stacked S4 on the merged S3 ingest (epic/36-surfaces-bake). S4 applies the stored scale and measures deviation; it does not invent a fitter or a report-only baker.

The baker now:

  • Fills each declared grid node (x_knot, y_knot) from samples whose X and Y equal that knot (f64 equality with f64::from of the u16). Missing nodes and two samples claiming the same node with different values are closed BakeErrors. Off-knot samples do not fill a node. Knot lookup is binary search on the strictly increasing lists.
  • Quantizes with round(value * scale) to i32, nearest, exact half-way away from zero. The host helper splits trunc and fraction so the f64 immediately below 0.5 stays 0 (adding 0.5 then flooring would emit 1). Overflow outside i32 is a closed BakeError. Zero / non-invertible scale is rejected at quantize. S3's non-finite sample/scale gate is unchanged.
  • Measures deviation of the quantized table from every supplied sample. MAX_ERR_LSB is ceil of an exact rational residual in crates/surfaces-bake/src/bound.rs (IEEE f64 bit-patterns as dyadics; bilinear as an exact ratio of the i32 grid on the host via num-rational BigRational). ceil applies only to the finished residual. Exact u16 coordinates also include the runtime-rounded integer path (evaluate_u16). Host f64 lerp is not the bound. A finite residual whose ceil does not fit in i32 is BakeError::BoundOverflow; NonFiniteDeviation is for a true non-finite residual. Exact-zero tables stay 0. Reviewed host crates are num-bigint, num-rational, and num-traits; they stay off the runtime graph. RMS is an operator statistic: Ratio::to_f64 uses a 53-bit window plus a binary exponent so f64::from_bits(1) (2^-1074) stays finite, not NonFiniteDeviation.
  • Rejects NX * NY above MAX_GRID_CELLS (1_048_576) with closed BakeError::GridTooLarge before allocating the cell matrix.
  • Emits pub const MAX_ERR_LSB: i32 = …; via emit_max_err_lsb. Full --emit-rust is not in this PR.

Runtime README / crate-doc status points at baker quantization and MAX_ERR_LSB. Baker types stay out of the firmware rustdoc.

Wording is deviation between the supplied samples and the table built from them. No device, vendor, sensor, calibration, accuracy, timing, flash, or WCET claim.

Bound proof

A test builds ph_surfaces::BilinearSurface from the quantized i32 grid. Exact-u16 samples use evaluate.

  • Rounding counterexample — knots [0, 2], grid [[0, 1], [1, 2]], sample (1, 1, 1): host reconstruct 1, runtime evaluate 2. Bound is 1.
  • Lerp rounding: knots [0, 3], grid [[4, 13], [75, 94]], sample (1, 1, 32.77777777777778). Exact bilinear is 286/9; three f64 lerps make the residual < 1; exact residual emits bound 2.
  • Decimal off-knot: knots [0, 1], grid [[1, 2], [3, 7]], sample (0.1, 0.1) stays finite.
  • Tiny dyadic on a zero table emits bound 1. Tiny sample (0.5, 0.5, 1e-300) against a unit table also emits bound 1.
  • Opposite-signed reconstruct: knots [0, 1], rows [0, -1], extra sample (1e-300, 0.5, 1). Exact reconstruction is -1e-300; residual is slightly greater than 1; bound is 2.
  • Unaligned lerp: knots [0, 1], both rows [1, 2], extra sample (1e-300, 0.5, 2). Exact reconstruction is 1 + 1e-300; residual ceiling is 1.
  • Finite residual 3_000_000_000 is BakeError::BoundOverflow, not NonFiniteDeviation. The same overflow with a same-sign tiny on rows [0, -1] is also BoundOverflow.
  • Exact integer residuals emit n, not n+1. Exact-zero tables still emit 0.
  • RMS of a zero 2×2 plus off-knot 1e-200 is 1e-200 / sqrt(5), not 0. RMS of f64::MIN_POSITIVE / 2 is that value over sqrt(5), not 0. A 2×2 of f64::from_bits(1) plus an off-knot of the same value quantizes to zeros, bound 1, RMS 2^-1074, not NonFiniteDeviation.

Not in this PR

Verification

cargo xtask ci on 07c0575 (full profile, not release): all runnable checks PASS; baker line budget 1409 / 1600. SKIP: secret scan (gitleaks not installed). A skip is not a pass. This is not release evidence.

PR base is epic/36-surfaces-bake (S3 ingest is merged there), not main.

Open in Web Open in Cursor 

Fill each declared node from on-knot samples, apply the stored scale with
round-to-nearest (ties away from zero), and measure deviation of the
quantized table from every supplied sample. The maximum is a durable
i32-value-LSB const fragment for later emission; operator stats stay on
the CLI. Missing nodes, ambiguous nodes, non-invertible scale, and i32
overflow are closed BakeError variants.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli
swgiacomelli marked this pull request as ready for review August 25, 2026 17:36
@swgiacomelli
swgiacomelli changed the base branch from cursor/issue-39-s3-ingest-f22c to epic/36-surfaces-bake August 25, 2026 17:37
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eece7a42df

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Unrounded host bilinear can understate deviation at u16 off-knot samples
because each runtime interpolation step rounds. The bound now takes the
max of that host reconstruction and the integer X-then-Y path whenever
both sample coordinates are exact u16 values.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 47bacba037

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Comment thread crates/surfaces-bake/src/quantize.rs
Interpolate the i32 grid with weighted f64 mixes so extreme dequantized
endpoints cannot overflow v1 - v0 to NaN. Reject NX*NY above 1_048_576
with a closed GridTooLarge error before allocating the cell matrix.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1e6bce7830

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Validated axes are strictly increasing, so fill and segment location
can binary-search instead of scanning. A 65_536-by-2 grid with one
sample per node no longer does billions of linear comparisons.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 452aea5bef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Adding 0.5 then flooring maps the f64 immediately below 0.5 to 1.
Split into trunc and fraction so values just under a half stay on the
nearer integer, while exact halves still round away from zero.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

1 similar comment
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c0473f7502

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Comment thread crates/surfaces-bake/src/lib.rs
cursoragent and others added 2 commits August 25, 2026 18:58
mul_add can round a residual that is slightly above an integer onto that
integer, so a plain ceil understated the Codex 1+eps / next_down(1) case.
next_up on a nonzero magnitude makes that case emit n+1; exact-zero tables
stay 0, and a residual just below an integer does not jump an extra LSB.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
Coupled-edit for the new host quantize API: the firmware README and crate
docs name that ph-surfaces-bake quantizes a caller-stated grid and emits
MAX_ERR_LSB as deviation from supplied samples, without dumping baker
types into the runtime rustdoc.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c5fadb0da

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
cursoragent and others added 2 commits August 25, 2026 19:17
Three f64 lerps can round a reconstruction upward so the residual lands
just under an integer while the exact bilinear of the i32 grid is just
over it. For exact u16 coordinates, MAX_ERR_LSB now uses num/den of that
grid; fractional coordinates keep an outward lerp envelope. Exact-zero
tables stay 0.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
A u16-only rational plus an 8-ULP lerp envelope still treats host
float as the oracle for fractional coordinates. Decode IEEE
bit-patterns as dyadics, reconstruct the i32 grid as a ratio at every
sample, and ceil that residual. Drop the one-ULP ceil padding.

Co-authored-by: Cursor <cursoragent@cursor.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0e4be2195b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/quantize.rs Outdated
Comment thread crates/surfaces-bake/src/quantize.rs Outdated
cursoragent and others added 2 commits August 25, 2026 19:29
Unreduced add/mul overflowed i128 on ordinary 0.1 off-knot coordinates
because shared 2^55 denominators were multiplied again. Cancel gcd first.
A dyadic whose exponent is below -126 keeps m/2^126 so a finite 1e-300
sample still emits a bound instead of NonFiniteDeviation.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
…y finite.

Unreduced i128 n/d overflows on 0.1-class interpolants and treats 1e-300 as non-finite. Keep powers of two in the exponent, and tell implementing agents in AGENTS.md not to substitute host f64 or ULP envelopes.

Co-authored-by: Cursor <cursoragent@cursor.com>
@swgiacomelli

swgiacomelli commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

MAX_ERR_LSB is now ceil of n/d * 2^exp with a 256-bit numerator in crates/surfaces-bake/src/bound.rs. Please re-review that residual (not host f64, not ULP padding, not a 2^-126 stand-in).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e73087ae8a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/bound.rs Outdated
Comment thread crates/surfaces-bake/src/bound.rs Outdated
Aligning 1e-300 with a unit reconstruction needs a ~1049-bit shift, so
add returns the dominant term when checked_shl fails. A finite ceil that
does not fit i32 is BakeError::BoundOverflow, not NonFiniteDeviation.
Raise the baker implementation cap to 1,600.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 76a5a89441

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/bound.rs Outdated
Returning only the dominant operand when exponent alignment exceeds 256
bits understates ceil for 1 - (-1e-300). Opposite-signed reconstructs
keep the dominant term; same-sign exact integers round the bound up.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

Re-review 719c615: exponent-separated addends (ceil must not understate), and BakeError::BoundOverflow vs NonFiniteDeviation.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 719c615454

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/bound.rs Outdated
Comment thread crates/surfaces-bake/src/bound.rs Outdated
cursoragent and others added 2 commits August 25, 2026 21:23
A ceil-oriented shortcut inside lerp turned 1+1e-300 into 2 and
understated the residual of sample 2 against rows [1, 2]. Carry the
unaligned addend through reconstruction; ceil of the two-term residual
must not understate, and a finite 3e9+tiny still maps to BoundOverflow.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
The baker may take reviewed std crates, so MAX_ERR_LSB can ceil an exact finite residual without a firmware-shaped 256-bit add and without treating overflow as non-finite.

Co-authored-by: Cursor <cursoragent@cursor.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f648b0bf9a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/Cargo.toml
Comment thread crates/surfaces-bake/src/quantize.rs Outdated
The coupled baker-dep trail missed a direct crate, and squaring 1e-200 underflows the operator statistic without changing MAX_ERR_LSB.

Co-authored-by: Cursor <cursoragent@cursor.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 614e38e185

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/bound.rs Outdated
Shifting both BigInt limbs by max(bits)-1023 zeros 1/2^1023, so RMS of
MIN_POSITIVE/2 printed 0. Cap the shift by the smaller limb so that
subnormal ratio stays representable. This is not MAX_ERR_LSB.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 532a95791c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread crates/surfaces-bake/src/bound.rs Outdated
1/2^1074 cannot be a finite f64 numerator over a 2^1074 denominator, so
to_f64 returned infinity and a valid bake became NonFiniteDeviation.
Keep the smallest subnormal as the operator statistic, not the bound.

Co-authored-by: Steven Giacomelli <swgiacomelli@users.noreply.github.com>
@swgiacomelli

Copy link
Copy Markdown
Contributor Author

@codex review

@swgiacomelli
swgiacomelli merged commit 48a301f into epic/36-surfaces-bake Aug 25, 2026
1 check passed
@swgiacomelli
swgiacomelli deleted the cursor/issue-40-s4-quantize-03fb branch August 25, 2026 22:05

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 07c0575922

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

_ if n_shift > d_shift => return f64::INFINITY,
_ => return 0.0,
};
let x = (nf / df) * 2.0_f64.powi(exp);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply the exponent without underflowing first

When both ratio limbs require truncation and their shift difference is -1075, powi(exp) underflows to zero before the normalized nf / df factor can raise the result into the representable subnormal range. Fresh evidence beyond the earlier scale-1 fixture is a zero-quantizing 2×2 table whose node and off-knot values are all f64::from_bits(0x0c70_0000_0000_0001) with scale f64::from_bits(0x3040_0000_0000_0001): every exact residual is slightly above half of 2^-1074, so its f64 value and RMS are the smallest subnormal, but this expression returns zero and publishes rms_lsb = 0. Convert the normalized significand and exponent together so intermediate exponentiation cannot underflow.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants