Advanced mode, 18 new filters across 4 new passes, deps 1.9.0, and a colour-tagging fix - #71
Open
StuartCameronCode wants to merge 35 commits into
Open
Advanced mode, 18 new filters across 4 new passes, deps 1.9.0, and a colour-tagging fix#71StuartCameronCode wants to merge 35 commits into
StuartCameronCode wants to merge 35 commits into
Conversation
Adding filters to VapourBox mostly means adding *methods to existing passes*,
not new passes — so the method dropdown is where complexity will accumulate.
This makes advanced mode the lever that keeps it in hand.
Advanced mode was `bool _advancedMode` inside the panel's State: per-panel,
defaulting off, and reset on every collapse. That made it useless as a lever,
because an expert re-flipped it constantly and so nothing could be hidden
behind it aggressively enough to matter. It is now one persisted app-wide
setting (AdvancedModeService, `showAdvancedOptions`), surfaced both by the
switch in each panel and in Settings → General.
It is provided ABOVE the MaterialApp, not inside `home`. `showDialog` pushes
onto the MaterialApp's Navigator, so a provider below it is out of scope for
every dialog route — which is exactly how the Settings switch first rendered
as Provider's red error box instead of a control.
`advancedOnly` now also applies to a MethodDefinition, so one schema can offer
a curated list to everyone and the full set on request:
dehalo 7 -> 3 Fine Dehalo 2 is a follow-up pass, Edge Cleaner is
line-art niche, and both Vinverse methods were
already duplicated in Chroma Fixes
noise_reduction 4 -> 3 MCDegrainSharp is a specialist chain
crop_resize 3 -> 2 EEDI3 upscale is several times slower than NNEDI3
for a marginal gain
Deinterlace, Deblock, Sharpen and Color Correction are left alone — 2-3
genuinely distinct choices each, not variant sprawl.
visibleMethods() always keeps the *selected* method, advanced or not: a preset
can select one, and hiding it would both misreport the pipeline and hand
DropdownButtonFormField a value absent from its items.
Method descriptions rewritten to say when to choose one and how fast it is,
since that is the only guidance the dropdown shows.
The pass list is grouped into five stage headers, each showing how many of its
passes are on. The stages are labels over the existing pipeline order, never a
reordering. Not collapsed by default: at 13 rows that hides enabled passes and
costs more than it buys.
Guards, because each of these fails silently rather than loudly:
- pass_list_stages_test: a PassType missing from `stages` renders nothing at
all, and reordering rows would misrepresent the pipeline.
- filter_schema_curation_test: lints all 13 shipped schemas — first method is
never advancedOnly (it is the resolved default), something always survives
simple mode, every method has a description, and no schema exceeds 4 methods
in simple mode. That last one is what keeps this strategy self-enforcing.
- dynamic_filter_panel_advanced_test: the panel behaviour, including that a
dialog route can resolve the service.
380 tests pass, up from 317.
… on conflicts
Completes the plan for absorbing filters without adding complexity: after
curating the method dropdowns, the remaining levers were relevance, presets and
pass interaction.
Relevance (pass_relevance.dart). The honest way to shorten the pass list is to
shorten it for the file in hand, and detection already returns enough to do it —
scan type, height, codec and SAR all arrive before the list is drawn. A matching
pass gets a "Suggested" badge and the reason ("source is hard telecine (3:2
pulldown)", "anamorphic source (10:11) — check pixel aspect"); one that cannot
apply says so.
It is a hint and stays a hint: nothing is reordered (row order is pipeline
order), nothing is enabled or disabled, and ScanType.unknown stays silent
because detection failed and claiming either way is worse than saying nothing.
Restraint is the load-bearing property — a badge on nine of thirteen rows is
decoration, not a recommendation — so the nine passes detection cannot judge
(dirt, scratches, grain, halos, banding, colour) return neutral for every
combination of scan type, height and codec, and the tests bound both.
Presets named after the source, because the user knows they captured a DV tape,
not that it wants SMDegrain with a chroma-bleed fix. Four added: DV Camcorder
Tape, PAL DVD / Broadcast, Anime DVD, 8mm / Super 8 Film Scan. A preset costs no
UI complexity at all, which makes this the cheapest capability in the plan.
Two silent bugs in those presets, both caught by writing the tests first:
- ProcessingPipeline()'s default DEINTERLACES, because QTGMCParameters.enabled
defaults to true. The film-scan preset was quietly deinterlacing a progressive
scan, i.e. softening it for nothing.
- A `preset:` enum on its own is only a label. NoiseReductionParameters(preset:
light) leaves every threshold at its default, so "light" denoised exactly as
hard as "moderate". Fixed by using the fromPreset factory.
Both are now documented in the "Adding a New Built-in Preset" checklist.
Interaction advice (pass_advice.dart), which is the complexity that actually
bites once the list is long: sharpening the denoiser will undo, an FPS divisor
IVTC ignores, Vinverse with deinterlacing off. Rendered through the existing
WarningBanner. Every combination it mentions still renders, so none of it is
validation — and advice only ever attaches to an ENABLED pass, since a banner on
a pass the user is not using is how advisory UI gets learned-to-ignore.
422 tests pass, up from 380.
…stion Two follow-ups from the UI work. Deletes nine dead files (2,071 lines). The hand-written *_settings_panel.dart widgets were replaced by the schema-driven PassSettingsInline + DynamicFilterPanelCompact on 2026-08-02 and have been unreferenced since: each declares exactly one class, nothing imports any of them, and they were last touched in March. Removing them also clears the repo's only remaining analyzer ERROR — a non-exhaustive switch over NoiseReductionMethod in the noise reduction panel, which had been failing to handle mcDegrainSharp in code nothing rendered. `flutter analyze lib test` is now error-free. Groups the preset menu, which had grown to nine built-ins in one flat list. "How hard should it try" and "what did you capture" are different questions, so they are now separate sections: "For Your Source" (six) and "Quality Only" (the three tiers, which do nothing but deinterlace). The source presets come first, because naming the source is the question a user can actually answer. The grouping key is a `category` on the preset, declared by each factory rather than looked up from a list of ids elsewhere — so a new preset cannot silently land in the wrong group. It defaults to `custom`, which is correct for user-saved presets and for presets already on disk without the field, and a built-in that forgets to declare one files itself under the source presets rather than vanishing. processing_preset_test asserts every built-in declares one, and that the pre-field JSON still loads. 427 tests pass, up from 422.
… HQDeringmod
Five filters from the Hybrid gap analysis, chosen for being both widely
regarded and genuinely low risk: every plugin was already sitting in the deps
bundle unused, so none of this needs a deps release. Each fails differently from
what its pass already offered, which is the only justification for another entry
in a method dropdown.
Noise Reduction DFTTest frequency-domain; the cleanest option on fine,
even grain
FFT3DFilter the traditional first stop for VHS luma noise
TTempSmooth very gentle temporal finish; leaves motion alone
Sharpen aWarpSharp2 warps pixels toward edges instead of raising
contrast, so it adds no halos
Dehalo HQDeringmod masked ring removal; protects the edge itself
The three denoisers are advancedOnly — SMDegrain still covers the common case,
so simple mode does not grow. aWarpSharp2 and HQDeringmod are visible: each is a
different mechanism rather than a variant, and Dehalo's own name already covers
ringing. Dehalo is now at the 4-method simple-mode cap, so the next addition
there has to displace something, which is the cap working as intended.
Constraints normalised in the worker rather than left to the plugin, because
each is accepted-but-wrong rather than an error: DFTTest's tbsize is forced odd
(an even window isn't centred on the current frame), FFT3D's bt and aWarpSharp2's
blur are clamped to their implemented ranges, and TTempSmooth's mdiff is held
below thresh (equal or greater silently disables the motion protection the
parameter exists for). HQDeringmod passes nothing it wasn't given, so havsfunc's
own tuning applies.
Two things went wrong, and both are now documented in CLAUDE.md:
KNLMeansCL was in this batch and was dropped. Probing the bundle showed its
OpenCL path does not initialise everywhere — the app's own knlm-probe.json
reports false on this Mac — CI deliberately excludes OpenCL-only plugins from the
required-namespace list, and its channels="YUV" mode requires 4:4:4, which none
of this app's sources are. It looked like a one-line win and was not low risk.
aWarpSharp2 first shipped with chroma=4, Avisynth's "warp chroma with the luma
mask". This VapourSynth port accepts only 0 or 1 and rejects anything else at
script evaluation, killing vspipe. Script generation passed throughout; only the
heavy end-to-end test caught it. `chroma` is now deliberately not passed at all —
asserted from both suites — matching how every other optional plugin argument
here is treated. A port does not necessarily share its Avisynth vocabulary.
Tests, both suites as required:
- 12 numbered Rust tests (test_94..test_103), including one that asserts each
method emits only its own filter call — a missed remove_block would chain two
denoisers silently — and one pinning the serde wire names, since a mismatch
falls back to the default instead of erroring.
- 7 script-generation tests in the push gate, 7 full-encode tests plus a
preview-render test in the heavy nightly set. Preview and encode are separate
scripts and separate ffmpeg invocations, so both are asserted.
435 push-gate tests pass (up from 427), 22 heavy, 341 Rust.
Second batch from the gap analysis. Unlike the first, these are whole
CATEGORIES the app had nothing in, so they are passes rather than methods:
Anti-Aliasing daa, santiag stair-stepping left by deinterlacing and
upscaling
Stabilize Stab shake and weave — telecine wobble, film scans,
handheld footage
Chroma Fixes LUTDeRainbow rainbowing (cross-luminance), the companion to
the LUTDeCrawl already there
All three come from havsfunc and MVTools, already in the bundle, so again no
deps release.
Two orderings carry meaning and are asserted from both sides (Rust test_110,
Dart pass_list_stages_test): anti-aliasing runs BEFORE sharpening, because
sharpening a stair-stepped edge makes the stepping more visible rather than
less; and stabilisation runs LAST before framing, because it shifts the picture
within the frame and exposes thin empty edges that a crop can then remove (or
that its own `mirror` argument can fill).
Normalised in the worker because each is accepted-but-wrong rather than an
error: santiag's `type` is pinned to nnedi3 (havsfunc also takes eedi2 and
sangnom, NEITHER of which is bundled — naming one fails at script evaluation);
Stab's shift limits are clamped non-negative (a negative value silently
disables that axis); and LUTDeRainbow gets LUTDeCrawl's convert-down-and-restore
guard, because it shares the same 8-10 bit limit.
Three things went wrong, all now in CLAUDE.md:
STPresso was in this batch and was dropped. havsfunc implements it with
core.flux.SmoothT and the fluxsmooth plugin is not bundled — zsmooth provides
FluxSmoothT under a different namespace. That makes it effort 2, not 1.
Stab shipped briefly with a `range` argument the bundled havsfunc does not have
(Stab(clp, dxmax, dymax, mirror)), so every job using it died with a TypeError.
The earlier probe called haf.Stab(clip) with no arguments and passed, proving
only that the function exists. inspect.signature is the check that catches this
— the same lesson as aWarpSharp2's chroma, one level deeper. `mirror` is now
exposed in its place, which is genuinely more useful.
And the heavy suite spent a full cycle testing a STALE WORKER BINARY. cargo test
compiles src/ into its own test executable, so the Rust suite passed against new
code while the Dart tests exec'd an old worker/target/debug/vapourbox-worker.
The symptom is misleading: unsubstituted {{PLACEHOLDER}} in the generated script
and a bare Python SyntaxError from vspipe, which reads like a template bug.
WorkerHarness now warns loudly when the binary predates worker/src or
worker/templates.
449 push-gate tests pass (up from 435), 27 heavy, 243 Rust. The one Rust failure
in `cargo test` is pre-existing: subtitle_integration_test needs the whisper
model, which CI provisions and this machine does not have.
The first filters in this series that need a DEPS CHANGE rather than something
already sitting in the bundle. fluxsmooth adds three Noise Reduction methods:
FluxSmoothT averages a pixel with its neighbours in time only where they
bracket it in value, so motion is left alone almost for free
FluxSmoothST as above plus the eight spatial neighbours
STPresso caps how far any pixel may move, so detail survives intact —
dropped from the previous batch precisely because it calls
core.flux.SmoothT internally and the plugin was missing
Deps 1.8.0 -> 1.9.0. Beyond the usual filter wiring this needed: a build block
in all three download-deps-* scripts, an entry per platform in
deps-expected-plugins.json, the namespace in vapoursynth_integration_test's
required list, and the version/tag bump.
Two constraints shaped the choice of plugin, both now in CLAUDE.md.
Windows has no from-source build path — download-deps-windows.ps1 only fetches
published release archives — so a plugin is addable only if upstream ships a
Windows binary, and every platform must then pin the version Windows can get.
FillBorders and Bwdif were the first two candidates and were rejected on exactly
this: their newest Windows binaries are several releases behind their source
(FillBorders v2 vs v4, Bwdif r4.1 vs r5.1), and pinning everything back that far
would have cost features that exist only in the newer source. fluxsmooth's
newest release v2 ships win32/win64, so there is no skew.
And it is built by invoking the compiler directly on its single C file rather
than through its autotools build. Adding autoconf/automake/libtool to three CI
deps workflows for one plugin is a poor trade; one line of cc gives identical
output with no new toolchain.
Verified on macos-arm64 by building the plugin, dropping it into the bundle and
running the filters end-to-end — all three encode and preview correctly, and
STPresso works, which confirms the earlier diagnosis of why it had to be
dropped. Windows and Linux are covered only when CI builds them.
*** THIS BRANCH NOW POINTS AT A DEPS RELEASE THAT DOES NOT EXIST YET. ***
deps-version.json names deps-v1.9.0. Before merge, follow the rc flow in
CLAUDE.md: cut deps-v1.9.0-rc1 as a prerelease, run the three build-deps-*
workflows against it, point deps-version.json at the rc on this branch, verify,
then publish the final deps-v1.9.0 and repoint. Scripts/release.sh already
refuses to cut an app release while deps-version.json names an -rc tag.
452 push-gate tests pass (up from 449), 30 heavy, 248 Rust.
The code for fluxsmooth landed in 35c63fe; this records the judgement behind it, which was only in the review discussion. zsmooth already provides FluxSmoothT/ST, so those two methods never needed the new plugin — they call the canonical flux namespace only because it is there. STPresso is the only filter that actually required it, since havsfunc hardcodes core.flux.SmoothT and cannot see zsmooth's equivalent. So the deps release buys exactly one filter. The alternative was considered and rejected: repoint the FluxSmooth methods at zsmooth, drop STPresso, revert deps-version.json to 1.8.0, and the whole branch would need no deps release at all. Keeping it was the call — STPresso is well regarded, the plugin is 34 KB, and the release is a one-time cost. Written down so the next person finding a 34 KB plugin behind a version bump does not have to work this out again, and knows that removing it means removing STPresso.
…othLevels
Batch four, all effort 1 — every plugin was already in the bundle, so no deps
change:
Noise Reduction CTMF large-window median for blotches and dropouts
Deblock DCTFilter ringing and mosquito noise, which the two
block-edge deblockers do not touch
NEW pass Film Grain AddGrain + GrainFactory3, to stop a denoised
picture reading as plastic and to hide banding
NEW pass Rotate / Flip sideways phone footage, mirrored captures
Color Correction SmoothLevels the same curve as Levels but dithered, so
stretching a narrow range does not band
Four parallel read-only agents probed every candidate against the real bundle
before any wiring was written. That is the only reason this batch works — they
found seven traps, none of which documentation would have shown, and three of
them were already live in code I had written:
A QUARTER TURN CHANGES THE PIXEL FORMAT. std.Turn90 swaps the chroma subsampling
axes, so 4:2:2 becomes 4:4:0, which vspipe emits as C440 and ffmpeg rejects
outright; 4:1:1 becomes a format with no y4m identifier at all, killing vspipe
itself. Both are hard job failures and 4:2:2 is the common 10-bit ProRes case.
The template now captures the source format and converts back.
SAR MUST BE INVERTED ON A TURN, IN TWO PLACES — the ffmpeg-side declaration and
{{SOURCE_SAR}} in the square-pixel fitting path. I had done the first and missed
the second.
TURNING INTERLACED MATERIAL IS UNRECOVERABLE. Fields are alternating rows; a
quarter turn puts them in alternating columns, where SeparateFields returns two
"fields" that each still contain both. No deinterlacer can fix it afterwards and
_FieldBased still claims the clip is fine. Now warned about in pass_advice.
CTMF REJECTS 9-BIT, and 9-bit is reachable — pixel_format.rs rounds odd depths
up through it. Guarded. Its memsize is also pinned to 16 MiB: at the plugin
default, 16-bit radius 3 measures 0.79 fps against 42, for identical output.
DCTFILTER ACCEPTS NaN AND BLACKENS THE FRAME — its range check is
`f < 0.0 || f > 1.0`, both false for NaN. The worker builds its eight factors
and guarantees they are finite. Its mapping is also separable
(factors[u] * factors[v]), not the max(u,v) of the Avisynth filter of that name.
grain.Add's `var` must NOT be depth-scaled, unlike every other level in this
app — it is already in 8-bit units and the plugin rescales internally.
SMOOTHLEVELS' DEFAULT CANNOT RUN: havsfunc calls core.f3kdb.Deband and this
bundle ships neo_f3kdb. useDB is pinned False. Its levels are also read in the
clip's own range (six arguments scaled in-script), and it crashes outright when
input_low > 0 and 1/gamma is not an integer, so the worker drops the black point
for that combination and the schema says so.
TemporalDegrain2 was requested and is NOT included. Its upstream repo declares
NO LICENCE, so vendoring ~4,300 lines of it is a legal decision rather than a
technical one and needs your call. It is also not effort 1: five modules not
one, postFFT=5 aborts the process rather than raising, postFFT=4 is broken two
ways, extraSharp is a NameError at exactly 16-bit, and depth-dependent limits
make outputStage=0 a complete no-op at >=12-bit while 73% of the degraining is
silently lost at 16-bit. bm3d is never reached, so it needs no deps addition.
Full findings in the task list and CLAUDE.md.
470 push-gate tests pass (up from 452), 36 heavy, 293 Rust.
…ow detail
Two more effort-2 filters, joining the ALREADY-PENDING deps 1.9.0 rather than
forcing another bump — that tag is still unpublished, so it was free to grow.
Chroma Fixes Bifrost temporal rainbow / dot-crawl removal. Where the
LUTDeRainbow added earlier decides within a frame,
this compares across frames, so it catches
rainbowing that shimmers as the picture moves
Color Correction Retinex lifts detail out of the shadows of underexposed
footage, by comparing each area to its
surroundings rather than raising the black level
Screening candidates against the Windows-binary rule is now the first step of
any effort-2 work, and it disqualifies about half of them. Of the ten checked,
Bifrost v3.0, Retinex r4, MiniDeen v2, MSmooth v1.1 and Descale r8 ship a
Windows binary on their newest release; DeDot v3, FillBorders v4, EEDI2 r7.1,
EdgeFixer r3 and TDeintMod r10.1 do not, and pinning every platform back to an
older tag to match would cost features that only exist in the newer source.
Both chosen plugins link nothing beyond system libraries, so they are safe on
all five bundles.
Two integration constraints, both measured against the bundle rather than read:
BIFROST IS 8-BIT ONLY — "Only constant format 8 bit integer YUV input
supported", confirmed failing at 10/12/16-bit and 4:2:2. It gets DeScratch's
convert-down-and-restore guard. Low impact in practice, since the composite
captures it targets are 8-bit anyway. Measured on an alternating-chroma clip it
halves the frame-to-frame chroma swing.
RETINEX REJECTS SUBSAMPLED FORMATS OUTRIGHT, and every source this app handles
is 4:2:0 or 4:2:2. Rather than round-trip the whole clip through 4:4:4 and
resample chroma twice for what is a brightness operation, the luma plane is
extracted as greyscale, processed and put back — colour comes through
bit-identical. Verified working that way at 8/10/12/16-bit and 4:2:2.
One build note worth keeping: bifrost includes <vapoursynth/VapourSynth4.h>
rather than <VapourSynth4.h>, so -I"$VS_INC_DIR" is not enough. The scripts
stage a small include root with a vapoursynth/ subdirectory and pass its parent.
Like fluxsmooth it is a single C file compiled directly, so no autotools are
added to CI; retinex is an ordinary meson build resolving headers via
pkg-config, which build_plugin already points at the from-source install.
deps-v1.9.0 still needs cutting before merge — now with three plugins rather
than one. The rc flow in CLAUDE.md is unchanged.
472 push-gate tests pass (up from 470), 38 heavy, 299 Rust.
ci-test.yml and nightly.yml could only ever download the bundle named by deps-version.json from a published release, so a deps change could not be tested until it was published -- and publishing an untested bundle is the thing you were trying to avoid. The documented way out was a prerelease rc tag, which works but is publicly visible. Both workflows now take an optional deps_run_id workflow_dispatch input. When set, .github/scripts/fetch-deps-bundle.sh pulls the zip from that build-deps-* run's artifact instead of from a release: private to the repo, no tag, and nothing to clean up afterwards. Draft releases were the obvious alternative and are the wrong one -- not for the reason CLAUDE.md gave (CI does authenticate, so the "neither CI nor the app can fetch them" claim was only ever true of the app), but because draft assets need push access, and getting there means widening the token to contents: write on a workflow that runs against pull requests. The artifact route buys the same privacy for a read-only scope, so both workflows declare an explicit read-only permissions block -- the repo's restricted default grants contents+packages read and nothing else, so actions: read has to be asked for or the fetch 404s. The rc flow is kept and documented for the one case artifacts cannot cover: an installed app has no token, so testing the real first-run download path still needs a published prerelease. The script prints only the zip path on stdout and hard-fails with the artifact listing when nothing matches, rather than letting an empty path surface later as a confusing unzip error.
One CI dispatch runs all four platform jobs, but macOS, Windows and Linux are three separate build-deps-* workflows with three separate run IDs, so a single run ID left three of the four jobs unable to find an artifact. The input now takes a comma-separated list and each job takes the first ID that holds an artifact for its own platform, which also means order doesn't matter and the same list works for every job. A run that built a different platform has no matching artifact and gh exits non-zero for it -- expected here, so the loop keeps going rather than treating the first miss as fatal.
All three were wired from the macOS script's vocabulary, and Linux does not
share it, so every one of them failed there -- and retinex failed on macOS
too. Caught by the deps-expected-plugins packaging guard, which is exactly
what it is for: three missing .so files rather than a bundle that installs
and then dies at job time with "No attribute with the name flux exists".
Linux:
- fluxsmooth and bifrost referenced $VS_INC_DIR, which is the macOS
script's name for it; Linux calls it $VS_INCLUDE_DIR and the unset
variable reached cc as a bare -I ("missing path after -I"). Both now
assert the variable is set, so a future rename fails saying which
plugin and which variable rather than emitting a confusing cc error.
- retinex was copied over without $PLUGIN_BUILD_ENV. macOS has no such
prefix, but on Linux it carries PKG_CONFIG_PATH, without which meson
cannot see VapourSynth at all.
macOS:
- retinex includes <vapoursynth/VapourSynth.h>, so pkg-config finding
VapourSynth is not sufficient -- the include root needs a CHILD
directory named vapoursynth. Linux has kept a symlink farm for this for
years; macOS had none, so bifrost staged a private include tree and
retinex, which resolves headers through pkg-config and cannot be handed
one, had no way to work at all.
macOS now mirrors Linux's farm, so a single -I"$VS_INC_DIR" satisfies
both include styles. bifrost drops its private staging, and retinex
needs no build-command change. The farm also warns if no API3 header
turns up to link: R78 installs only the API4 set and both scripts top
up VapourSynth.h from the source tree, so retinex silently depends on
that top-up having happened.
They were written into the arm64 side of the plugin arch split, so the x64 job never reached them and the packaging guard failed on all three missing dylibs -- the same symptom as the Linux failure, a different cause, and invisible on arm64 where everything passed. x64 takes pre-built binaries for most plugins, but Stefan-Olt ships none of these three, so they have to come from source on both arches. They now sit after the split, which is where zsmooth already lives for the same reason. Nothing else is needed to make that work on x64: MACOSX_DEPLOYMENT_TARGET is exported near the top of the script, so the two direct cc calls inherit the 12.0 floor and pass the minos guard (issue #39), and build_plugin sets PKG_CONFIG_PATH from VS_PC_DIR itself, which is resolved before the split. arm64 is unaffected -- it built all three successfully at the previous commit and the move only changes which branch they are reached from.
…Windows
The SmoothLevels path suppresses the plain std.Levels call with
replace("core.std.Levels(\n clip,\n)", "clip")
which is a literal two-line pattern. A Windows checkout gives the .vpy
templates CRLF endings, so the pattern never matched there and BOTH the
plain Levels call and SmoothLevels survived into the generated script --
silently, because a missed replace leaves valid-looking Python rather than
an error. Caught by test_132's negative assertion, on Windows only.
Fixed at the root: load_template_by_name normalizes CRLF to LF on read, so
the whole class goes away rather than this one pattern. It is currently the
only multi-line pattern in the generator, but it was not obviously the last.
Python does not care about the endings.
templates_load_with_lf_endings_only asserts both templates load without CR
on every platform, so the guard fails everywhere rather than only on the
platform that has the problem.
Also records in CLAUDE.md what these three plugins actually cost: four red
deps builds, each a different cause, all from copying a block between
download scripts that do not share a vocabulary.
znedi3 refuses a clip carrying _FieldBased and fails the whole job with "Failed to retrieve frame 0 with error: znedi3: _FieldBased". This pipeline sets that property whenever a field order is known -- including with deinterlacing OFF, where it is deliberately kept so a later resize resamples chroma field-aware -- so "anti-alias with deinterlacing off" was a hard failure on every bundle whose interpolator is znedi3. It passed on macOS arm64 (nnedi3 via havsfunc patch 6) and on Windows (whose prebuilt znedi3 accepts it), and failed on macOS x64 and Linux x64. That is the worst shape for a bug: the same job either worked or died depending on the user's machine. Anti-aliasing operates on whole frames, so the pass now clears the mark and restores it immediately afterwards -- restoring matters, because a later resize still needs it, which is the entire reason it is set when deinterlacing is off. Both templates, kept in step, or a preview would fail where the encode succeeded. Found by the nightly heavy suite. The script-only gate cannot see this class at all: it asserted haf.daa( appears, which it did, in a script that could not run. test_139/test_140 assert the guard's shape and that a source with no detected field order still generates exactly what it did before.
…not rejects it My previous commit had the cause backwards and did not fix anything: it cleared _FieldBased only when a field order was KNOWN, while the failing case is the opposite one. Measured against the bundled znedi3 rather than inferred this time: field=1 no property: OK =0: OK =2: OK field=3 no property: ERROR =0: OK =2: OK havsfunc's daa calls nnedi3 with field=3, and the pipeline only sets _FieldBased when a field order is known — so an ordinary source with none (the heavy suite's fixture, and most real sources) reached znedi3 with the property absent and failed the whole job with "znedi3: _FieldBased". macOS arm64 uses nnedi3 (patch 6) and Windows ships a prebuilt znedi3; both tolerate the absence. macOS x64 and Linux x64 build znedi3 from source and do not. Hence the same job working or dying by machine. The pass now always marks the clip instead of depending on an upstream pass having done it: 0 once deinterlacing has run (that output IS progressive) or when nothing was detected, and the detected order otherwise, which daa needs to pair fields correctly. Stamping the source's original order back on after deinterlacing would tell every later filter the progressive output is still interlaced, so that distinction is asserted rather than assumed. test_139 covers the case that actually failed (no detected order), test_140 that deinterlacing yields 0, test_141 that BFF is not flattened.
A terse plugin error naming a property says the property is involved, not in which direction -- reading "znedi3: _FieldBased" as a rejection produced a fix that changed nothing, when znedi3's double-rate mode in fact requires the property. Records the probe table, the fix, and that two platforms passing was not evidence: arm64 (nnedi3) and Windows (prebuilt znedi3) both tolerated the absence while the two from-source bundles did not.
StuartCameronCode
force-pushed
the
feat/advanced-mode-and-filter-curation
branch
from
August 17, 2026 01:33
1b15aab to
4ee577a
Compare
… found Seven parallel read-only agents probed all 25 unshipped Core/Strong candidates from the gap analysis against deps/macos-arm64 before any plan was written. The plan itself, the simple-vs-advanced calls and the preset defaults live in the artifact; this commit records what the probing invalidated in here. Probing changed the verdict on nine of the twenty-five. The four durable lessons, each of which would have cost a cycle: - A forum consensus about AviSynth is not evidence about this pipeline. TIVTC was top-rated on "TIVTC is definitely better for complex DVDs", repeated across VideoHelp. Against the repo's own hard_telecine_test.avi it is bit-identical to the vivtc path already shipping — 0.0000/255 after matching and after decimation. Measure the premise before pricing the work. - Two candidates were already implemented. EDI upscaling is complete in pipeline_template.vpy and measured within 0.055 px of a reference resample; it is invisible only because its schema section is advancedOnly/expanded:false. GrayWorld is not a second filter — in YUV it reduces exactly to AutoWhite. - Upstream defaults can be no-ops or hard failures. zsmooth.Cnr4 defaults scenechange=True and needs frame props this pipeline never sets, so a naive call fails 100% of jobs on every platform. Checkmate is the mirror image: at its own default tthr2=0 it measured 0.000 difference on every pattern tested. - ReduceFlicker's non-SIMD path reads the wrong neighbour frame and aarch64 has no other path, so the ARM bundles would have rendered differently from x86 — the znedi3 _FieldBased shape again. Corrections to rules this file states: - The Windows-binary rule is intact but three of its conclusions were stale, because upstream moved distribution channel rather than stopping. Bwdif r5 and DeDot v3 both publish PyPI wheels for all five targets, and EdgeFixer r3 does ship a Windows asset. The check is now two commands, and the PyPI one was missing; the akarin block in download-deps-windows.ps1 is already a working wheel fetcher to copy. A wheel's macOS tag is a floor, not a promise — dedot's macosx_15_0_x86_64 still fails the 12.0 STRICT_MIN_OS guard. - test_93 scans only the two .vpy templates, not vendored .py modules. It has never mattered because spotless.py uses no Expr, but TemporalDegrain2 and mClean both do, so vendoring either would silently take the 21x scalar interpreter path on ARM. - 8 of the 16 shipped passes are used by no built-in preset at all. Stabilize shipped specifically for film scans and the 8mm preset does not use it; CCD is the filter VideoHelp prescribes most for tape and is in no preset, including VHS Cleanup. Audit the presets whenever a pass ships. New section: colour metadata is dropped end to end and is NOT yet fixed. Matrix, primaries, transfer and range are never read (ffprobe returns them; the parser discards them), never carried on either VideoJob, and never stamped — build_ffmpeg_args emits no -color* flag at all, so every output is untagged and read as BT.601 limited. Same shape and same fix site as the SAR bug. A SetFrameProps pass would be inert: the Y4M pipe strips frame properties too. The preview has a companion bug — it hardcodes in_range=tv with no in_color_matrix while the "before" thumbnail does see the real tags, so a 709-tagged source shows a hue shift no filter caused.
Second seven-agent probe round, this time over every remaining Useful-tier candidate from the gap analysis. One promotion out of twenty-two — MVTools FlowFPS as a Frame Rate pass — which is the expected shape for a tier defined as "second answers", not a disappointment. The plan, curation calls and preset defaults are in the artifact; this records the transferable lessons. - A "new" filter is often the shipped one with different arguments. KillerSpots measured bit-identical to spotless.py (max diff 0.0) with three mvtools arguments changed — the third instance after lostfunc.DeSpot and GrayWorld. Those arguments are better: spot MAE 10.49 -> 9.33 at 318 -> 424 fps, so a three-line change to shipped code is worth more than the filter was. - An automatic filter must be tested on the material it should ignore. AutoDeblock's detection is inverted: on blocked MPEG-2 it altered the picture less than it altered clean footage, and on grainy-but-unblocked content it fired strong on 99% of frames. Any "auto" filter gets a three-way test, and the clean cases matter more than the damaged one. - Check candidates against the content this app's own presets create. FillDrops cannot distinguish a dropped frame from a held animation cel — both are bit-exact — so it destroyed 39 of 80 held frames at every threshold. We ship Anime DVD and DVD IVTC presets where duplicates are normal. - Prefer a crash you can catch. vs-placebo constructs its node with no exception when Vulkan is absent, then segfaults on the first frame, taking vspipe down for both job and preview with nothing to detect. Worse than KNLMeansCL, which at least raises. - FrameMap::Retime already exists and no pass emits one. FlowFPS's output count matches it exactly across 35 combinations where BlockFPS is off by one in 14, so choosing FlowFPS makes existing code correct as written. Also records a probe finding that did NOT survive verification, because the correction is the useful part: an agent reported SMDegrain as a near no-op at the app's defaults. Its repro omitted RefineMotion and prefilter, which the template always emits. With the real defaults it removes 3.34 of 4.36 grain and the reachable parameter space is well-behaved across 27 combinations. No bug — but bare SMDegrain on uniform synthetic noise finds perfect motion matches everywhere and gates everything out, so validate MC denoisers on real footage.
Colour metadata was dropped at all three stages. It was never read — ffprobe returns color_space/color_transfer/color_primaries/color_range and the parser took width, height, pix_fmt, SAR and field order and discarded them. It was never carried on either VideoJob. And it was never stamped: build_ffmpeg_args emitted no -color* flag of any kind. So every file this app wrote was untagged, and an untagged file is read as BT.601 limited by every player — silently shifting the colours of any BT.709 or full-range source. Measured end to end through the worker on a bt709 + full-range clip: source bt709, pc output before unknown, tv output after bt709, pc This is the same bug as the SAR one, with the same cause and the same fix site. A SetFrameProps pass in the script would be inert — the Y4M pipe strips frame properties exactly as it strips SAR, verified — so the tags are output-stream flags on the encoder, emitted immediately after the setsar block. Values are validated rather than forwarded. ColorMetadata::from_raw drops anything not on FFmpeg's accepted list, on the same principle as parse_ratio: ffprobe reports the literal string "unknown" for an untagged stream, and forwarding that would fail the whole encode on an argument the user can neither see nor fix. Each tag is independent, so a source declaring only a matrix gets only -colorspace, and an untagged source stays untagged rather than being guessed at. Also fixes a companion bug in the preview. It hardcoded `-vf scale=in_range=tv:out_range=pc` with no in_color_matrix, so swscale guessed the matrix, while the app's "before" thumbnail comes from a separate ffmpeg call on the original file that does see the real tags. On a 709-tagged source the comparison showed a hue shift no filter had caused, and a full-range source was range-stretched twice. build_ffmpeg_args_for_test duplicates build_ffmpeg_args rather than calling it, which is how the SAR block came to be missing from it. The colour block is mirrored there with a comment saying why. Tests: 8 unit tests on the mapping, 3 on the emitted arguments, 4 Dart tests on the ffprobe side, and a heavy end-to-end round trip in integration_chroma_subsampling_test.dart — the only level that can prove ffmpeg honoured the flags, since these are encoder arguments a generated-script test cannot see.
StuartCameronCode
changed the title
Eight of the sixteen shipped passes were used by no built-in preset at all. Every batch added capability that nothing enabled, so a user picking a preset named after their source got none of the work done for that source. The four that matter, all existing passes and no new code: - 8mm / Super 8 Film Scan gains Stabilize. This pass shipped specifically for film scans two batches ago and no preset used it, while gate weave is the first thing anyone notices on a cine scan. The most obviously incomplete preset in the set until now. - VHS Cleanup gains Chroma Denoise (CCD) and Sharpen (LSFmod) — the two filters the restoration community prescribes most for tape, both shipping unused while this preset ran SMDegrain and DeHalo_alpha alone. Chroma noise on VHS is a different failure from luma grain, so the denoiser did not cover it. LSFmod at 80 rather than the default 100: it runs after a denoise, and sharpening a denoised capture hard is how tape ends up looking artificial. LSFmod is the right sharpener here because it is built not to add halos, which matters when the dehalo pass has just removed some. - DV Camcorder Tape gains Chroma Denoise, gentler than VHS. DV chroma is misaligned more than it is dirty, and the light luma denoise deliberately leaves chroma alone. - Anime DVD gains LUTDeRainbow, which shipped for composite-mastered discs rainbowing on fine line art and was used by nothing. Restraint is the point: no preset gains everything available to it, and the three quality tiers are unchanged, because a preset is a claim about a source. processing_preset_test.dart asserts each of these against what the preset's name promises — that file is where two silent preset bugs were already caught. Also bounds the tape sharpening strength rather than pinning it, since the failure mode is oversharpening after a denoise. Verified end to end: the rewired VHS Cleanup pipeline (QTGMC + SMDegrain + CCD + DeHalo_alpha + LSFmod) encodes cleanly.
…e EDI upscaling Two Phase 1 items from the plan, both already in the bundle. Cnr4 (zsmooth) joins CCD in the Chroma Denoise pass. They are complementary rather than alternatives: CCD smooths blotches that sit still, Cnr4 settles colour that swims or shimmers between frames. Measured at 1015 fps, so it is effectively free. Visible rather than advanced — two methods is not a crowded dropdown, and the distinction is one a user can act on. Two traps, both measured against the bundled plugin and both fatal: - scenechange defaults to True and requires the _SceneChangePrev/Next frame properties, which this pipeline never sets. A bare Cnr4(clip) therefore fails 100% of jobs on every platform. misc.SCDetect now runs in front, which is the better fix than scenechange=False because it is also what stops the filter smearing chroma across a cut. - It rejects 4:1:1, and 4:1:1 is NTSC DV, which pipe_source maps natively. The template converts up to 4:2:2 and restores the source format afterwards, the same shape as the DeScratch bit-depth guard. The plugin takes sense and str per plane. Exposing three numbers for what reads as one idea is how a pass stops being usable, so one slider drives all three and preserves the plugin's own luma:chroma ratio. EDI upscaling was already fully implemented — rpow2 doubling with per-plane centroid correction, measured within 0.055 px of a reference resample — and invisible, because the whole section was advancedOnly with expanded:false. Split into a visible "Upscale" section with the three controls needed to reach the feature, and an advanced "Upscale tuning" section for the ten network knobs. This is the highest-value curation change in the plan and cost no new code. Not touched: crop_resize declares methods[] that nothing reads, the real choice being the upscaleMethod parameter. Removing it means teaching the curation lint to tolerate a schema with no methods, which delivers nothing to users. Verified end to end against the real plugin, both methods, plus generation tests from Rust and Dart. 482 Dart tests, 145 Rust integration tests.
… white balance Three more Phase 1 items, none of which needs a plugin. ContraSharpening is a checkbox inside Noise Reduction rather than a Sharpen method, and it has to be: it takes the pre- AND post-denoise clip, so it cannot sit in the linear pass chain. The template captures the input at the top of the pass and applies the restore at the bottom. Phrased in the UI as the outcome — "restore detail after denoising" — because unlike a sharpener it cannot invent detail that was not there, so it is safe to leave on when denoising hard. Note the placeholder is NR_POST_CONTRASHARP, not NR_CONTRASHARP: SMDegrain already has its own contrasharp argument using that name. Auto levels and auto white balance are ~40 lines of PlaneStats and need no plugin at all. GrayWorld is not a third thing — in YUV the grey-world assumption reduces exactly to shifting the chroma plane means onto neutral, which is what auto white balance does, so it ships as one control. Two things the probing changed about the naive implementation: - Auto levels measures its stats on a heavily downscaled copy, not the full frame. Min and max of a full frame are set by single outlier pixels, so one specular highlight re-grades the whole shot — and because those outliers come and go, the result flickers frame to frame. Averaging them away first is what makes it usable on real footage. - PlaneStatsMin/Max come back in the clip's own depth and are used raw; only the user's targets, which are 8-bit UI units, are scaled to the format. Getting that backwards is the bug class that already shipped twice. PlaneStatsAverage is normalised at every depth and must not be scaled at all — the auto white balance path relies on that. Auto white balance guards num_planes < 3: PlaneStats(plane=1) errors on GRAY. Its Expr goes through the _expr() helper, so ARM takes the akarin JIT rather than the per-pixel interpreter. Verified end to end against the real plugins: ContraSharpening over SMDegrain, and auto levels + auto white balance together.
Two independent items. Both were promoted on measurement rather than on the strength of the idea. FRAME RATE (MVTools FlowFPS) — the one promotion out of 22 Useful-tier candidates. Scoped deliberately to standards conversion, not smoothing: interpolating a master to a higher rate invents frames that were never photographed and makes the file a worse record than the tape. Converting an already-converted PAL/NTSC tape is the opposite case, where the damage is in the source and doing nothing means judder or a 4% speed error. So it offers named target rates rather than a free number, ships off by default, and is never described as making motion "smooth". FlowFPS rather than BlockFPS, and the reason is arithmetic rather than quality. FrameMap::Retime already existed in the worker and no pass emitted one. Measured over 35 combinations, FlowFPS produces floor(n_in * ratio) — exactly what Retime::output_count computes — while BlockFPS produces floor((n_in-1)*r)+1 and is off by one in 14 of them, which would desynchronise the progress total and the preview index. Choosing FlowFPS makes existing code correct as written. Three things worth knowing: - The ratio is reduced before it reaches FrameMap. 25 -> 29.97 is 1200/1001, not the plugin's 30000/1001; Retime multiplies a frame count by that pair, so an unreduced ratio overflows on a long clip. - The source rate is carried on the parameters, because the pipeline cannot see the job and a wrong fixed ratio is worse than none — same reason inputSar is carried rather than looked up. Absent, it reports Identity. - mvtools rejects 4:1:1 at Super, and 4:1:1 is NTSC DV, precisely the tape an NTSC->PAL conversion targets. Guarded up to 4:2:2 and restored. The pass runs genuinely last of the video passes, asserted from both sides. pass_relevance stays silent on it: a 25 fps source is not evidence anyone wants 29.97, so this is a delivery decision detection cannot make. HISTOGRAM — app-side, computed from the preview PNG the app already holds. No plugin, no deps release, no worker round trip, and it updates as a slider moves rather than costing a re-render. Chosen over the VapourSynth histogram plugin, which has no LICENSE file at all. Forum advice on levels is literally "watch the histogram", and the app had colour controls with no way to see what they were doing — now more so, having just gained automatic levels and automatic white balance that a user needs to be able to sanity-check. Binning is a pure function over RGBA bytes so it can be tested without a widget: 13 tests including all-black, all-white, a known 4-pixel image asserting exact luma bins, and a 256-level ramp. Rec.709 luma in 8.8 fixed point with weights summing to exactly 256, so neutral grey lands in its own bin with no drift. Scaling uses the interior peak, ignoring bins 0 and 255 — a letterboxed frame piles enormous counts into bin 0 and scaling to that flattens the real shape to nothing.
…he bundle
Five plugins into the still-unpublished deps 1.9.0, unblocking the Phase 3
filters. All five verified locally on macOS arm64 — built or downloaded through
the actual new script text, loading, and producing a frame on a real clip.
Two of these were previously rejected and the rejections had expired, which is
the finding worth keeping: the Windows-binary rule is intact but upstream
projects have been changing distribution channel rather than stopping.
- Bwdif: GitHub assets stop at r4.1 (2021), but r5 moved to PyPI and
vapoursynth-bwdif 5.1 ships wheels for all five targets at minos 10.15/11.0 —
under both macOS floors, so unlike akarin it costs the Intel bundle nothing.
- DeDot: v2/v3 ship no GitHub assets; vapoursynth_dedot 3.0 ships wheels for all
five. BUT both macOS wheels are minos 15.0 against the Intel floor of 12.0
under STRICT_MIN_OS, so macos-x64 builds from source instead. A wheel's macOS
tag is a floor, not a promise.
- FillBorders stays pinned to v2, the last tag with binaries, because the v2/v4
difference vanishes at even border widths — which is the only thing the UI
will offer.
- RemoveDirt and LGhost take Stefan-Olt prebuilts where they exist and build
from source on linux-aarch64.
Three platform-specific traps handled, all from the four-check table:
- macOS: the whole section sits after `fi # end plugin arch split`, so x64
reaches it. A block inside the arm64 branch is invisible on Apple Silicon and
simply never runs on Intel.
- Linux: $VS_INCLUDE_DIR, not macOS's $VS_INC_DIR, and meson needs the
$PLUGIN_BUILD_ENV prefix. Both asserted with ${VAR:?} so a rename fails naming
the plugin.
- Windows: RemoveDirt-1.1.7z ships x64\ and x64_Clang\ copies of the same DLL
name, and the existing filter matched both, making the winner copy-order
dependent. The extraction loops now take an optional Prefer regex, pinned to
x64_Clang.
Attribution taken from upstream and quoted, per issue #72. FillBorders has no
copyright line anywhere in its repo and DeDot names no author for the AviSynth
original — NOTICES records both gaps rather than inventing a name. Bwdif and
LGhost vendor VCL2, so both join the existing Vector Class Library entry.
Not verifiable here: the Windows script (no pwsh on this machine, reviewed by
hand), Linux x86_64 wheel load, and the macOS x64 minos guard on a real Intel
runner. Those are CI's to confirm.
"Is there a faster way to deinterlace than QTGMC?" is a perennial question and
the app had no answer: every tier was a cheaper QTGMC. Bwdif measures 622 fps
against QTGMC Fast's 150 on the same clip — 4-5x — at most of the quality.
Visible rather than advanced, and it takes the deinterlace schema to exactly the
4-method simple-mode cap. A speed tier is a goal a user can act on, not a
variant of something already offered.
Three measured facts shaped the wiring:
- No format limits at all. 8/9/10/12/16-bit, 4:2:0/4:2:2/4:4:4, GRAY, RGB and
even 4:1:1 all pass, so unlike almost every recent addition it needs no guard
in either template.
- No _FieldBased sensitivity: field=1 and field=3 both work with the property
absent, 0, 1 or 2. It does not repeat the znedi3 double-rate trap that cost
two nightly cycles.
- It accepts an external interpolator itself, which is why Yadifmod is not a
separate method. Measured, Bwdif+nnedi3 (0.524) beats Yadifmod+nnedi3 (0.532)
at the same cost, so the whole of that candidate collapses into one advanced
checkbox here. EEDI3 as the interpolator is 11x slower for no gain and is
deliberately not offered.
`field` is required, not optional, and its parity half comes from the same TFF
value QTGMC uses — deriving it separately is how the preview and the render come
to disagree about field order. Its FrameMap mirrors QTGMC's: Fanout{2} at double
rate, Identity otherwise, but radius 1 rather than 3, because it reads one
neighbour each side rather than a temporal window.
Verified end to end on a real interlaced fixture: both interpolators, and double
rate correctly producing 42 frames from 21 at twice the rate.
…r dot crawl Two Phase 3 filters, both using plugins added to deps 1.9.0 in the previous commit. REMOVEDIRT ships as a second SpotLess method, and the case for it is speed rather than quality. Measured against the shipped SpotLess on 80 frames with synthetic single-frame spots: SpotLess spot MAE 9.21 clean 0.222 143 fps RemoveDirt spot MAE 16.9 clean 0.278 908 fps RemoveDirtMC spot MAE 9.99 clean 0.222 202 fps So the motion-compensated variant is deliberately NOT offered: it lands on the same quality point as SpotLess and runs slower, adding nothing. The plain form gives 6.3x the throughput for about 60% of the removal, which is what makes it usable on a long capture where SpotLess would take hours. Two deviations from the canonical chain, both measured: - No trailing RemoveGrain(17). It is the single largest source of collateral damage in the chain — on its own it takes clean-pixel MAE from 0.19 to 0.70 and touches 29% of pixels. Dropping it is worth 3.2x less damage at identical spot removal and 36% more speed. Exposed as an off-by-default option instead. - The Clense family comes from zsmooth, not rgvs: rgvs.Clense raises at 9-14 bit and float, zsmooth's equivalents work at every depth this app can produce. The plugin rejects 9-bit and 4:1:1, and both are reachable here — pixel_format rounds odd depths up through 9, and 4:1:1 is NTSC DV — so both are guarded and the source format restored. Verified at 8/9/10/16-bit, 4:2:0/4:2:2/4:1:1 and GRAY. Its noise thresholds are normalised internally (bit-identical output at 8/10/12/16-bit), so unlike most of this codebase they must not be depth-scaled. DEDOT joins Chroma Fixes as a toggle beside LUTDeCrawl and LUTDeRainbow, and it is complementary rather than a replacement — measured on a line-alternating crawl LUTDeCrawl reached 0.00 residual where DeDot left 11.69, while on a phase-inverting checkerboard DeDot reached 0.24/0.00 on both planes where LUTDeCrawl left chroma untouched. Different geometries, so both earn their place. It is 8-bit only, so it gets the DeScratch convert-and-restore guard with error diffusion on the way back up. Both verified end to end, and the classic SpotLess path re-verified unchanged.
…Degrain2 Five new modules in worker/templates/, not yet wired into the pipeline. Also extends test_93 to scan them, and fixes a struct literal I broke in 2062db9. DEFLICKER — two methods, and the plugin is deliberately not used. ReduceFlicker's non-SIMD path reads prevp[0]/[2] where its SIMD path correctly reads nextp[0]/[2], and the SIMD block is #if defined(__SSE2__), so aarch64 has only the buggy path — the ARM bundles would have rendered differently from x86, the same shape as the znedi3 trap. Transcribed to Expr instead and validated against a numpy model of the C semantics: max |diff| 0 levels across all six strength/aggressive combinations. The global method is the more valuable one and it beat its own brief: 83.5% of injected flicker removed against a 75% target. Both a gain and an offset term are fitted because neither alone is good on all fault types — gain-only manages 46% on additive flicker, offset-only 46% on multiplicative, the affine fit is 1.45-1.62 levels residual on all three. AUTOCHROMAFIX — reimplemented rather than copied; the reference is unlicensed. Recovers an injected whole-pixel chroma shift exactly: 42/42 at ±1px and 24/24 at ±2px across 4:2:0 and 4:2:2 at 8/10/12/16-bit. 0.07s once with a reference frame, then 1020 fps. Two findings from getting there. Resampling each candidate shift is what breaks whole-pixel accuracy — a shift landing on a whole chroma sample resamples nothing while a half-sample one is maximally softened, and that ripple drags the peak 0.25px. Scoring integer lags as crop offsets, with no interpolation anywhere, fixed it. And signed gradient correlation has a content-dependent sign, so the directional gradient is rectified; std.Prewitt's combined magnitude is too flat to use. It also refuses to invent a shift when chroma carries no measurable edge structure — a score curve spanning under 5% of its peak reports zero rather than guessing. The VHS-ish fixture is exactly that case and correctly gets no correction. MCLEAN and TEMPORALDEGRAIN2 are vendored from Selur/VapoursynthScriptsInHybrid, which has no LICENSE file and no headers. Proceeding on an explicit decision to accept that; each header records the upstream repo, file and line range, states the licence position plainly, and carries the upstream docstring attributions verbatim. No licence or copyright holder is invented anywhere. Four TemporalDegrain2 guards, all measured against the bundled plugins: postFFT is clamped to 0-3 because postFFT=5 aborts the process rather than raising (a zero block size reaches FFT3DFilter); grainLevel is clamped because the autotune tables are length 6; extraSharp's NameError at exactly 16-bit is fixed at its source in MinBlur; and the depth-dependent limits are scaled from clip.format, which takes 8-vs-16-bit parity from 2.85 to 0.45 on the FFT sigma and from 0.77 to 0.22 on the mvtools limit. Without that last one the filter lands at 2.005 against a 2.0 test tolerance. mClean pins three parameters: deband is repointed to neo_f3kdb and clamped to 0-1 (the level-2 path reaches an unbundled plugin), icalc is forced True (the float path needs absent mvsf), and outbits is forced to the source depth so the downstream chroma conversion is not handed a format it did not expect. test_93 now scans every worker/templates/*.py, not just the two .vpy templates. That gap never mattered because spotless.py uses no Expr — these modules do, and without the check they would silently take the 21x scalar interpreter path on ARM. Verified it fails on all three violation shapes.
Three new passes, taking the pipeline from 17 to 20. Each fills a category the app had nothing in. DEFLICKER answers the complaint every cine-film thread is about, and which the shipped "8mm / Super 8 Film Scan" preset could not touch. Two methods because they address different faults: the global one fits a gain and an offset per frame against a windowed neighbourhood average and removes 83.5% of injected flicker; the local one damps per-region oscillation a whole-frame statistic cannot represent. Global is the default — cine flicker is a whole-frame exposure fault. Both terms are fitted in the global method because neither alone works everywhere: gain-only manages 46% on additive flicker, offset-only 46% on multiplicative, the affine fit 1.45-1.62 levels residual on all three. EDGE REPAIR rebuilds the dirty rows tape captures leave at the frame border, which until now could only be cropped away — throwing picture away with them. FillBorders won a measured three-way: its error is constant at 1.29 across three damage models, the signature of a filter that discards the border and rebuilds from the interior, so the result cannot depend on how bad the damage is. EdgeFixer is luma-only and would leave the coloured fringe that makes a tape edge obvious; bbmod lost every case and costs 19 parameters. Widths are even and that is load-bearing, not cosmetic: the bundle pins FillBorders v2 as the newest tag with binaries, and v2 is bit-identical to v4 at even widths, differing only at odd ones where it leaves subsampled chroma unrepaired. Crop already steps by 2 for the same reason. interlaced=-1 reads _FieldBased so a known field order is handled without a second control. GHOST REMOVAL cancels the displaced echo RF and cable distribution leave behind — a distinct, frequently reported tape complaint with nothing addressing it. LGhost was the cleanest plugin probed for this work: no format limits found at any depth or subsampling, and a clean _FieldBased matrix. Its natural interface is a repeating (mode, shift, intensity) triple, unlike any control here, so simple mode gets a short strength preset and the triple editor sits behind advanced. The worker drops any entry the plugin would reject — mode 0 and intensity 0 are hard errors at script evaluation, and the three arrays must stay the same length, so filtering has to be simultaneous. Pipeline order is asserted from both sides. Edge repair precedes every spatial filter and the resize; ghost removal precedes the denoise so the echo is not averaged into the picture; deflicker follows deinterlacing, because field-doubled frames break every temporal comparison it makes. `opt` is deliberately not exposed on LGhost: measured on arm64 every value gives byte-identical output, so it is inert here and a footgun on x86. All three verified end to end, including an invalid ghost being dropped and odd edge widths rounding down.
…gnment The last of the Phase 2 filters, now connected to their vendored modules. mCLEAN takes Noise Reduction's final simple-mode slot, and that is a deliberate call the curation lint now pins. It is the only candidate in the whole gap analysis that is a *goal* — denoise, restore detail, restore grain, behind one control — rather than another mechanism, which is exactly the distinction the lint exists to enforce. The schema is now at the 4-method cap and cannot grow again without something moving behind advanced. TEMPORALDEGRAIN2 is the most-requested filter in the analysis and stays behind advanced mode anyway: 21 fps, a dozen interacting parameters, and three values that break it outright. An expert asks for it by name; a novice should never land on it by scrolling. The worker clamps postFFT to 0-3 in addition to the module's own clamp — 4 and 5 abort the process rather than raising, so neither layer should be the only one. AUTOMATIC CHROMA ALIGNMENT joins Chroma Fixes as the counterpart to the manual shift sliders that were already there. It measures the misalignment instead of asking the user to guess it, recovers an injected whole-pixel shift exactly across 4:2:0/4:2:2 at 8-16 bit, and costs 0.07s with a reference frame. It refuses to invent a shift when chroma carries no measurable edge structure, which matters because a soft VHS chroma channel is exactly that case. One bug caught by test_115, worth recording because the shape recurs. I added the two new remove_block calls by appending them to an existing line — but that line does not appear in the arm where its own block is the active one, so STPresso alone never stripped the new blocks and left unsubstituted placeholders. A missed remove_block chains two denoisers silently: valid VapourSynth, twice the runtime, not what was asked for. Every arm is now checked programmatically rather than by pattern-matching on a sibling.
The second half of the preset work. Phase 0 wired the passes that already shipped; this wires the ones added since. - Fast now uses Bwdif. Measured 622 fps against QTGMC Fast's 150 — until now this tier was only a lower QTGMC preset, which is not what the name promises. The QTGMC preset is left in place so switching method in the UI lands somewhere sensible. - VHS Cleanup gains Edge Repair at two rows all round, and DeDot. Dirty edge rows are near-universal on tape and cropping them throws picture away; dot crawl along sharp colour edges is the signature composite artefact and nothing in the preset touched it. - 8mm / Super 8 Film Scan gains Deflicker. With Stabilize from the earlier commit it now addresses both faults cine film actually has — gate weave and brightness pulsing — where before it addressed neither. - DV Camcorder Tape gains automatic chroma alignment. DV's problem is chroma above all, and measuring the shift beats asking the user to guess it. - Anime DVD gains DeDot alongside LUTDeRainbow. Complementary rather than redundant: measured, each reaches a crawl geometry the other leaves alone. Every one asserted in processing_preset_test.dart against what the preset's name promises, including that the edge-repair widths are even — odd widths are the only case where the pinned FillBorders v2 differs from v4.
Phase 4, and with it every filter in the plan is implemented. BURN-IN draws a supplied .srt or .ass into the picture using libass, which is compiled into all four bundled ffmpeg builds. Deliberately scoped to a user-supplied file: Whisper transcription runs AFTER the encode, so its output does not exist when the encoder needs it. Moving transcription before the encode is a larger change and a separate piece of work — the two burn-in subtitle modes fall back to writing the sidecar rather than silently doing nothing. This required fixing a latent bug first. The -vf slot was single-use: the code appended either setsar or setdar, and ffmpeg takes the LAST -vf and silently drops earlier ones. Appending a subtitles filter would therefore have thrown away the aspect stamp — breaking the third leg of issue #50 in a way nothing would have caught. Filters now accumulate into one chain, verified: a 16:11 anamorphic source with burnt-in subtitles comes out still tagged 16:11. The path is per job, not an encoding setting. A path held in settings would apply one file's subtitles to every video in a batch. CUSTOM VAPOURSYNTH is the escape hatch, on the same footing as the Custom FFmpeg Arguments that already existed — arbitrary local execution is not a new risk in a process that already loads arbitrary plugins. The real hazard is the frame count, and it is guarded rather than trusted. A snippet calling Trim or SelectEvery changes the true output length while the declared total stays put, which makes the progress bar lie and makes frame-accurate preview show a different frame than its label — both silently. The generated script now captures len(clip) before the snippet and refuses afterwards if it changed, naming both numbers and pointing at the trim controls. It also asserts clip is still a VideoNode, since the commonest mistake is forgetting to assign the result back. Verified: a benign snippet runs; core.std.Trim is refused with "changed the frame count from 11 to 3". Also caught by schema_converter_integration_test, which asserts enum values match schema options — the two new subtitle modes had to be added to the schema, not just the enum. That test earned its place.
Six lessons from implementing the plan, each of which cost time or would have shipped a silent bug: - The -vf slot was single-use and ffmpeg silently drops all but the last one, so adding subtitle burn-in would have thrown away the aspect stamp. - Adding a remove_block by pattern-matching on a sibling line misses the arm where that block is the active one. - Whisper burn-in is a different feature, not a harder one: transcription runs after the encode. - The hazard in custom user code is the frame count, not the arbitrary execution, because it corrupts progress and preview silently. - FrameMap::Retime already existed unused; check before adding a variant, and reduce the ratio before handing it over. - A wheel's macOS tag is a floor, not a promise.
test_144 asserts each new pass emits its own call and leaves no unsubstituted
placeholder of its own — prefix-scoped rather than a bare {{ search, because
the template's docstring documents the placeholder syntax with literal
{{PARAMETER_NAME}} examples.
test_145 pins that edge repair rounds every width down to even. That is not
cosmetic: the bundle pins FillBorders v2, which is bit-identical to v4 at even
widths and differs only at odd ones, where it leaves subsampled chroma
unrepaired.
test_146 pins the custom-VapourSynth frame-count bracket, in both directions —
present when a snippet is supplied, and the whole block removed when it is not.
Seventeen filters became twenty-one. Adds the four new passes — Edge Repair, Ghost Removal, Deflicker and Frame Rate — and notes Bwdif as the fast option on the Deinterlace row. Also rewrites the cleanup summary, which listed what the filters address rather than what a user has: it now names dirty capture edges, aerial ghosting and cine-film flicker, and says the presets turn on what a source needs, since that is now true where before it largely was not.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
20 commits, 93 files, +12.3k/-2.4k. The branch grew well past its original scope, so here is the whole of it.
The throughline: VapourBox gains a lot of Hybrid's capability, and a mechanism to stop that turning it into Hybrid. Every filter here is reachable in one click, and none of them lengthens a dropdown a beginner sees.
deps-v1.9.0is still a draft release.app/assets/deps-version.jsonon this branch points at it, so once an app release references that tag it must be published or every user's first-run download 404s.Scripts/release.shonly refuses-rctags, so it will not catch a draft. All five platform zips and their.sha256.jsonsidecars are already staged on the draft; CI has been green against them via the newdeps_run_idroute.The complexity lever
This is the load-bearing design decision, and it is why 18 filters can land without the UI getting worse.
AdvancedModeService— advanced mode wasbool _advancedModeinside one panel's state: per-panel, off by default, and reset on every collapse. That made it useless as a lever, because an expert re-flipped it constantly, so nothing could be hidden behind it aggressively enough to matter. It is now one app-wide setting (Settings → General), provided throughMultiProvider, gatingadvancedOnlysections, preset-controlled parameters andadvancedOnlymethods.filter_schema_curation_test.dart— lints every shipped schema: the first method is neveradvancedOnly(it is the resolved default), at least one survives simple mode, every method carries a description, and no schema offers more than 4 methods in simple mode. That last assertion is the point: adding methods is expected, letting a dropdown grow unchecked is what it catches. Curated down: dehalo 7→3, noise_reduction 4→3, crop_resize 3→2.What's new
4 new passes — Anti-Aliasing, Stabilize, Rotate/Flip, Film Grain — taking the pipeline from 12 to 16.
18 filters in five batches, all of them plugins or functions that were already in the bundle and unreachable except where noted:
4 new presets, named after the source rather than the technique — PAL DVD / Broadcast, DV Camcorder Tape, 8mm / Super 8 Film Scan, Anime DVD. A user knows they captured a DV tape; they do not know it wants SMDegrain with a chroma-bleed fix. Presets are the cheapest capability in the app — they cost no UI complexity at all.
Suggestions and advice, both deliberately toothless — neither blocks a job, disables a control or changes a value.
pass_relevance.dartbadges passes that suit the loaded file;pass_advice.dartcomments on pass combinations (sharpening a denoiser will undo, an FPS divisor IVTC ignores). The load-bearing property is restraint: passes detection cannot judge stay silent, and the tests bound how many badges a source may produce.deps 1.8.0 → 1.9.0
Adds fluxsmooth, bifrost and retinex. Only STPresso actually required a plugin — havsfunc hardcodes
core.flux.SmoothTand cannot see zsmooth's equivalent — and that call was made deliberately; bifrost and retinex joined the same unpublished tag rather than forcing another bump.Also
deps_run_id:ci-test.ymlandnightly.ymlnow accept aworkflow_dispatchinput namingbuild-deps-*run IDs, so an unpublished bundle can be tested straight from a run artifact. This removes the chicken-and-egg that previously forced rc prereleases, and nothing about it is publicly visible.Bug fixes
build_ffmpeg_argsemitted no-color*flag at all. So every file this app wrote was untagged, and an untagged file is read as BT.601 limited by every player. Measured end to end on a bt709 + full-range source:unknown, tv→bt709, pc. Same shape as the SAR bug, same fix site. ASetFramePropspass would have been inert — the Y4M pipe strips frame properties exactly as it strips SAR. Fixed a companion preview bug in the same change, wherein_range=tvwas hardcoded with noin_color_matrix, showing a hue shift no filter had caused.daadied on macOS x64 and Linux x64 withznedi3: _FieldBased. Read as "znedi3 rejects field-based clips" it produced a fix that changed nothing, because the truth is the opposite: znedi3's double-rate mode requires the property, and this pipeline only sets it when field order is known. It survived on arm64 and Windows and died on the two bundles that build znedi3 from source — the worst shape a bug can have.load_template_by_namenormalises CRLF, but a two-linereplace()pattern stopped matching on a CRLF checkout, leaving both the plain and Smoothstd.Levelscalls in the script.Research behind it
47 filter candidates probed against the real bundle by parallel read-only agents before any wiring was designed. Full analysis, the plan for what is next, the simple-vs-advanced call per filter and the preset defaults: The Hybrid Gap.
Probing repeatedly beat reading. It found two candidates already fully implemented and merely hidden, two "dead" upstreams that had moved to PyPI wheels, a plugin whose non-SIMD path reads the wrong neighbour frame so ARM would have rendered differently from x86, one filter that is a complete no-op at its own upstream default and another that fails every job at its default. It also killed four filters that read well on paper: TIVTC measured bit-identical to the
vivtcpath already shipping.CLAUDE.mdcarries the traps that cost real time, including the ones that were nearly shipped.Testing
cargo test— 171 + 163 + 143 unit/integration, including numbered filter tests for every new pass and parameterflutter test --exclude-tags heavy— 480 passingReviewing
The commits are readable in order and each is self-contained. If you only read three:
8b00be5(the lever and why per-panel state was useless),94537d5(suggestions/advice and why they stay hints),4b9d4b0(the colour fix, with the measurement).Note
app/lib/**/*.g.dartis gitignored, so the four newinputColor*fields exist in generated serialization only afterdart run build_runner build. CI regenerates; a stale local copy silently drops the fields.