Skip to content

vello_gpu: Fix methods for bilinear and bicubic image sampling - #1963

Draft
laurenz-canva wants to merge 3 commits into
laurenz/reduce_slotsfrom
laurenz/fix_bilinear_image
Draft

laurenz-canva wants to merge 3 commits into
laurenz/reduce_slotsfrom
laurenz/fix_bilinear_image

Conversation

@laurenz-canva

@laurenz-canva laurenz-canva commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

This PR makes the following changes:

  • It adds a shim in the test suite so that we can run most external texture tests (the once that sample the full texture) with Vello CPU as well. By doing so, Vello CPU can serve as the reference, and we increase the coverage of tests that compare against Vello CPU and GPU.
  • It fixes the issue where, instead of extending each tap of bilinear/bicubic samples, we only extended the original sample location. This is wrong, and is the reason why a couple of image tests were inconsistent between CPU and GPU. In addition to that, it also ports some changes to the extend logic that we recently applied to Vello CPU, to make the results more numerically stable.

This does unfortunately regress performance: Rendering a full-sized image on my Android phone goes from around 23FPS to 13FPS. See the below videos.

Before:

IMG_1296.MOV

After:

IMG_1297.MOV

But the previous approach was just fundamentally wrong. However, the good news is that with #1964, I will introduce a fast path that uses GPU-native bilinear sampling for images that are sampled from the whole external texture, which will not only undo this slowdown, but in fact make image rendering more than 2x faster compared to current main, when using external textures and sampling the whole texture! See that PR for more information. Since rendering whole images is the most common operation (except for glyph caching, but this is experimental right now, anyway. And nearest-neighbor sampling is less affected than bilinear sampling), in my opinion this is a trade-off worth taking.

Using the image atlas will unfortunately stay slow, even with #1964, but I think that's something we have to accept for now. We could improve this in the future by restricting the possible image sampling modes for such images (for example, only allowing extend mode Pad). But the problem is simply that when we sample from an arbitrary subregion of an atlas, we have to emulate correct extension all ourselves, which is much slower than letting the hardware do it, so this should be avoided anyway.

@laurenz-canva
laurenz-canva marked this pull request as draft September 29, 2026 14:58
@laurenz-canva
laurenz-canva added this pull request to stack #1965 September 29, 2026 14:59
@laurenz-canva
laurenz-canva force-pushed the laurenz/fix_bilinear_image branch from 8ae8442 to 690d88d Compare September 29, 2026 15:08
Base automatically changed from laurenz/size-impr to main September 30, 2026 06:18
@LaurenzV
LaurenzV force-pushed the laurenz/fix_bilinear_image branch from 690d88d to 93e02c4 Compare September 30, 2026 06:18
@@ -1,5 +1,6 @@
{
"timeouts": {
"pageLoad": 1200000,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to add this to address another timeout failure resulting from our newly added tests: https://github.com/linebender/vello/actions/runs/36677557384/job/109765766086

@laurenz-canva
laurenz-canva force-pushed the laurenz/fix_bilinear_image branch from f50d0ce to 4ee3473 Compare September 30, 2026 07:36

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The border in the new snapshot is expected. The test uses bilinear sampling with repeat. This means that for example, for the top-right part, as we approach the right border of the yellow pixel, it should slowly start to fade into the red of the top-left pixel, hence why it becomes orange.

@LaurenzV
LaurenzV marked this pull request as ready for review September 30, 2026 08:02
@LaurenzV
LaurenzV requested a review from grebmeg September 30, 2026 08:58
@LaurenzV LaurenzV changed the title vello_gpu: Implement proper bilinear and bicubic image sampling vello_gpu: Fix methods for bilinear and bicubic image sampling Sep 30, 2026
@LaurenzV
LaurenzV removed the request for review from grebmeg October 1, 2026 09:29
@LaurenzV
LaurenzV marked this pull request as draft October 1, 2026 09:29
@laurenz-canva
laurenz-canva force-pushed the laurenz/fix_bilinear_image branch from 4ee3473 to 0fa8fe9 Compare October 1, 2026 10:58
@laurenz-canva
laurenz-canva removed this pull request from stack #1965 October 1, 2026 10:59
@laurenz-canva
laurenz-canva changed the base branch from main to laurenz/reduce_slots October 1, 2026 11:00
@laurenz-canva
laurenz-canva added this pull request to stack #1969 October 1, 2026 11:00

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants