Skip to content

perf(r): reuse single-chunk nanoarrow vectors - #951

Draft
paleolimbot wants to merge 1 commit into
apache:mainfrom
paleolimbot:codex/fast-nanoarrow-vctr
Draft

paleolimbot wants to merge 1 commit into
apache:mainfrom
paleolimbot:codex/fast-nanoarrow-vctr

Conversation

@paleolimbot

Copy link
Copy Markdown
Member

Adds an as_nanoarrow_array.nanoarrow_vctr() fast path for full-slice, single-chunk vectors. Existing buffers are reused; a shallow array wrapper is created only when a data-frame field name must be attached. This avoids extension round-trips such as rebuilding GeoArrow geometry through WK.

On a 3.23M-row GeoArrow reproduction, data-frame-to-stream conversion fell from 3.44 s to 0.32 s. The available R test suite and R CMD check --no-manual pass.

library(nanoarrow)
x <- as_nanoarrow_vctr(as_nanoarrow_array(c("one", "two")))
as.data.frame(as_nanoarrow_array(data.frame(x = x)))
#>     x
#> 1 one
#> 2 two

@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.21%. Comparing base (8f81815) to head (cc80c4f).

Files with missing lines Patch % Lines
r/R/vctr.R 92.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #951      +/-   ##
==========================================
+ Coverage   78.19%   78.21%   +0.02%     
==========================================
  Files         106      106              
  Lines       16845    16870      +25     
  Branches     1985     1985              
==========================================
+ Hits        13172    13195      +23     
- Misses       2439     2441       +2     
  Partials     1234     1234              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants