Skip to content

Performance improvement for many rows - #6

Open
meld-cp wants to merge 1 commit into
Codeer-Software:mainfrom
meld-cp:main
Open

Performance improvement for many rows#6
meld-cp wants to merge 1 commit into
Codeer-Software:mainfrom
meld-cp:main

Conversation

@meld-cp

@meld-cp meld-cp commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Hi,

I'm not sure what your appetite is for this change but I was still having performance issues with my use case, here is the summary:

This change makes OverWrite much faster for large loops.

The old version repeatedly modified worksheet rows while processing each loop. The new version plans the final layout first, then applies row, style, and value changes in batches. This makes the code more complex, but avoids expensive repeated work in ClosedXML.

What Changed

  • Plans all output rows before editing the worksheet.
  • Adds rows in bulk instead of inserting them one at a time.
  • Copies rows and styles in groups.
  • Writes values in batches when possible.
  • Caches object-property lookups.
  • Avoids unnecessary asynchronous work for the built-in converter.
  • Preserves support for formulas, formatting, merged cells, paging, nested loops, and custom functions.
  • Adds coverage for nested scopes, custom converters, cached lookups, and large styled reports.

The public OverWrite APIs remain unchanged.

Performance

I used RecursiveLoop2LargeRowCountWithManyPropsTest with 10,000 source records. After one warm-up run followed by three measured runs the results were:

Commit Runs Median
This PR 330 ms, 361 ms, 484 ms 361 ms
ac016916 Did not finish within 60 seconds >60 seconds

The new version completed in under half a second. The baseline was still running after one minute, so the measured improvement is at least 150× faster. The actual speed-up is likely higher.

The trade-off is additional planning and materialization code. However, the benchmark shows that this complexity greatly reduces processing time when pushing many rows.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant