Batch 'addArticle's

[ISNIT0]

The majority of CPU time in CPP (18.8%) when using node-libzim seems to be calls to v8::internal::Builtin_HandleApiCall.

I'm not sure how much it would improve the performance, but we could try creating an addArticles method in CPP and automatically batch the adding on the JS side.
This would keep the same simple API, but potentially improve performance

[kelson42]

@ISNIT0 Have a look to nodejs/node#17996, might be completely insignificant.

[ISNIT0]

@kelson42 in that issue, the total percentage was 0.1%, in our case it's 18.8%, so I think it matters much more for us 😀

[kelson42]

@audiodude Do you have some kind of insight on this. Actually I had forgotten about this issue, and that might be a good way to speed-up MWoffliner.

[audiodude]

I'm not sure. I think mwoffliner will continue to be I/O bound because of all the network calls.

[kelson42]

This issue need to be refreshed. Do we have still a nice performance optimisation potential here?

[audiodude]

I had Claude do a benchmark with mwoffliner and node-libzim. The conclusions are below. I have read through them and they seem to generally make sense to me.

Yes I know, AI sucks, but I feel like this is a cheap way to provoke discussion. I don't think it replaces human judgement or should immediately drive decisions.

---

It seems the answer to @kelson42's question is: no,
there's no meaningful optimization left here, and the issue could be closed. The reason is what happened at the N-API
boundary, not just network-boundedness.

The 18.8% figure profiled a binding that no longer exists. In 2019, addArticle was a plain synchronous N-API method:
creator_->addArticle() ran inside the N-API call frame, on the JS main thread (old src/writer.cc), and the TS wrapper
awaited a non-promise. Every article's marshalling and cluster work blocked the event loop, so Builtin_HandleApiCall was
genuinely hot — and batching would have amortized it. The 3.0.0 N-API rewrite deleted that code path.

Today's N-API boundary is already async and nearly free. Creator.addItem queues a Napi::AsyncWorker and returns a
Promise (src/creator.h), so zim::writer::Creator::addItem runs on the libuv threadpool. The only main-thread work left is
a handful of property reads plus one V8→C++ content copy in the StringItem constructor — per-item V8 work that batching
cannot eliminate.

I benchmarked it (Ryzen 5800X, Node 22, libzim 9.8.1, 5000 × 49 KB items, zstd):

mode	per item	items/s
mwoffliner-style (mutex + await each)	4386 µs	228 (~11 MB/s)
serial await, no mutex	4401 µs	227
1-byte items — isolates the N-API call + AsyncWorker + Promise round-trip	14.4 µs	69 500
same, with the event loop artificially 50% busy	4493 µs	223

So the entire N-API overhead a batch addItems() could save is ~14 µs/item. The other 4.4 ms/item is libzim's internal
compression, zstd at level 19.

On the mwoffliner side, batching has nothing to bite on either: it passes StringItems exclusively (so libzim never
round-trips into JS for content), serializes writes behind a mutex that costs nothing measurable, and already pipelines
download vs. save — the code's own comment says "in normal circumstances, where downloading is slower than saving, this
promise will always be resolved here already" (saveArticles.ts). Per-article domino parsing/rewriting on the main thread is
milliseconds, ~100× the whole N-API boundary cost. Over a 7M-entry dump, batching saves ~100 seconds total, spread across
days.

If ZIM write throughput ever does become the constraint, the levers are configNbWorkers (mwoffliner leaves the default of 4)
and libzim's hardcoded zstd level.