perf: optimize spark_base64 in spark-expr

[andygrove]

⚠️ Autonomously generated by an LLM. This PR was created end-to-end by an automated agent with no human author. It has passed automated correctness (unit tests + differential fuzz) and micro-benchmark gates, but requires full human review. Do not merge without expert scrutiny.

What changed

Optimizes spark_base64 in the native spark-expr crate.

Reuse scratch String buffers for base64 encode + line-wrap and build directly into a preallocated StringBuilder, eliminating two per-row heap allocations in the array path.

Evidence

• Correctness: unit tests + seeded differential fuzz (bit-identical Arrow output vs main).
• Benchmark (criterion, Rust-only): see numbers below; gate required at least one benchmark ≥ 5.0% faster (non-overlapping confidence intervals) and no benchmark meaningfully slower.

spark_base64_ long, chunked: 21.073% faster (base 882215ns -> cand 696308ns)
spark_base64_ short, chunked: 11.932% faster (base 216853ns -> cand 190979ns)
spark_base64_ short, unchunked: 12.069% faster (base 215962ns -> cand 189896ns)
spark_base64_ long, unchunked: 11.43% faster (base 627480ns -> cand 555761ns)

[mbutrovich]

The scratch-buffer reuse is the real win and it stands on its own. One concern with the capacity precompute in encode_array:

let data_capacity: usize = (0..array.len())
    .filter(|&i| array.is_valid(i))
    .map(|i| base64_encoded_len(array.value(i).len()))
    .sum();

Two issues:

1. This adds a full O(n) pass over the array before the encode loop makes its own pass. For a change whose goal is removing per-row allocations, the extra walk is a cost that partly offsets the win.

2. base64_encoded_len returns the unwrapped length. When chunk is true (the Spark default), the output includes CRLF separators (about 2 bytes per 76 chars), so the builder under-reserves and can still reallocate on long chunked input. The PR description says it "sizes the value buffer up front", which only holds for the unchunked path.

I would drop the precompute entirely. The builder's amortized doubling is fine, and removing it saves the extra pass. If you want to keep it, make the estimate chunk-aware so the reservation is actually tight for the default path.

Separately, the Evidence block in this PR is clean (all four reported benchmarks exist in the added base64.rs), which is worth noting since the sibling perf PRs had reported numbers that do not map to committed benchmarks.

[comphead]

should we reserve every row? perhaps we can do a reusable buffer?