Summary
Implement a generic, modular benchmark execution engine capable of running standardized loading and query workloads across multiple triple store implementations and datasets.
Objectives and Technical Scope
1. Modular Driver SPI
• Define a generic store adapter interface (e.g. TripleStoreDriver) specifying lifecycle methods: init(), load(datasetPath), query(sparqlQuery), clear(), close().
• Support automated metric collection before, during, and after each phase (time, heap memory, GC stats).
2. Scenario Runner and Result Export
• Build a CLI/runner executing test matrices: [Store] x [Dataset] x [Query Workload] x [Repetitions].
• Export standardized execution results in machine-readable JSON and CSV formats for subsequent analysis and dashboard rendering.
Acceptance Criteria
[ ] Generic driver SPI is defined and fully decoupled from any single triple store implementation.
[ ] Automated benchmark runner executes configurable benchmark matrices.
[ ] Metric collector accurately captures ingestion speed, query execution latencies, and JVM memory delta.
[ ] Results are exported in structured JSON format.
Summary
Implement a generic, modular benchmark execution engine capable of running standardized loading and query workloads across multiple triple store implementations and datasets.
Objectives and Technical Scope
1. Modular Driver SPI
• Define a generic store adapter interface (e.g.
TripleStoreDriver) specifying lifecycle methods:init(),load(datasetPath),query(sparqlQuery),clear(),close().• Support automated metric collection before, during, and after each phase (time, heap memory, GC stats).
2. Scenario Runner and Result Export
• Build a CLI/runner executing test matrices:
[Store] x [Dataset] x [Query Workload] x [Repetitions].• Export standardized execution results in machine-readable JSON and CSV formats for subsequent analysis and dashboard rendering.
Acceptance Criteria
[ ] Generic driver SPI is defined and fully decoupled from any single triple store implementation.
[ ] Automated benchmark runner executes configurable benchmark matrices.
[ ] Metric collector accurately captures ingestion speed, query execution latencies, and JVM memory delta.
[ ] Results are exported in structured JSON format.