Summary
Provide an optimized batch ingestion API in core.next.storage to ingest high-volume RDF graphs without memory exhaustion or excessive garbage collection overhead.
Objectives and Technical Scope
1. Batch Ingestion Interface
• Design a high-throughput batch loading contract (addBatch(Iterable<Statement>) / stream consumer) in core.next.storage.api.
• Implement bulk index population that minimizes pointer re-allocations and tree rebalancing overhead.
2. Index Deferral and Rebuilding
• Support temporary index suspension or bulk insertion modes during massive loads, followed by deterministic index finalization.
Acceptance Criteria
[ ] Bulk loading of 100,000+ statements completes without memory exhaustion or GC stalls.
[ ] All index invariants (SPO, POS, OSP) are fully satisfied and consistent post-ingest.
[ ] Memory footprint per statement during batch loading is measurably optimized compared to one-by-one insertion.
Summary
Provide an optimized batch ingestion API in
core.next.storageto ingest high-volume RDF graphs without memory exhaustion or excessive garbage collection overhead.Objectives and Technical Scope
1. Batch Ingestion Interface
• Design a high-throughput batch loading contract (
addBatch(Iterable<Statement>)/ stream consumer) incore.next.storage.api.• Implement bulk index population that minimizes pointer re-allocations and tree rebalancing overhead.
2. Index Deferral and Rebuilding
• Support temporary index suspension or bulk insertion modes during massive loads, followed by deterministic index finalization.
Acceptance Criteria
[ ] Bulk loading of 100,000+ statements completes without memory exhaustion or GC stalls.
[ ] All index invariants (SPO, POS, OSP) are fully satisfied and consistent post-ingest.
[ ] Memory footprint per statement during batch loading is measurably optimized compared to one-by-one insertion.