Rewrite the benchmark harness from Groovy to Java 21, with a reproducible measurement protocol (JVM warmup, 3 repetitions, median, memory delta).
Motivation
The current Groovy code has several issues:
Unstable memory measurement (dependent on GC timing)
No repetitions → non-reproducible results
Cumulative measurement across files → cannot isolate the cost of a single file
No JVM warmup → first result biased by JIT compilation
French locale bug on decimal separators in CSV output
Objectives
- Generic TripleStoreDriver interface
public interface TripleStoreDriver {
void init();
void load(String filePath);
void query(String sparql);
long getGraphSize();
void clear();
void close();
}
- Measurement protocol
For each file:
warmup: load() → clear() → gc() // discarded
Repeat 3 times:
gc()
memory snapshot before
load()
memory snapshot after
gc()
→ median of 3 repetitions
→ memoryDelta = after - before
- Collected metrics
loadingTimeMs (median over 3 repetitions)
throughputTriplesPerSec
memoryDeltaMb (real delta, not global heap)
cpuPercent (via OperatingSystemMXBean)
- JSON output format
{
"metadata": {
"generatedAt": "...",
"protocol": { "repetitions": 3, "warmup": true }
},
"runs": [
{
"engine": "corese.4.6.5",
"dataset": "dbpedia",
"file": "category_labels.nt",
"triples": 565108,
"loadingTimeMs": 12310,
"throughputTriplesPerSec": 45890,
"memoryDeltaMb": 424.8,
"cpuPercent": 78.2
}
]
}
Files to delete
src/main/groovy/*.groovy
python-utils/plot-compare.py
python-utils/template.html
public/
Files to create
src/main/java/fr/inria/corese/benchmark/driver/TripleStoreDriver.java
src/main/java/fr/inria/corese/benchmark/harness/BenchmarkHarness.java
src/main/java/fr/inria/corese/benchmark/metrics/MetricsCollector.java
src/main/java/fr/inria/corese/benchmark/metrics/BenchmarkResult.java
src/main/java/fr/inria/corese/benchmark/Main.java
Update build.gradle for Java 21, remove Groovy
Rewrite the benchmark harness from Groovy to Java 21, with a reproducible measurement protocol (JVM warmup, 3 repetitions, median, memory delta).
Motivation
The current Groovy code has several issues:
Unstable memory measurement (dependent on GC timing)
No repetitions → non-reproducible results
Cumulative measurement across files → cannot isolate the cost of a single file
No JVM warmup → first result biased by JIT compilation
French locale bug on decimal separators in CSV output
Objectives
Files to delete
src/main/groovy/*.groovy
python-utils/plot-compare.py
python-utils/template.html
public/
Files to create
src/main/java/fr/inria/corese/benchmark/driver/TripleStoreDriver.java
src/main/java/fr/inria/corese/benchmark/harness/BenchmarkHarness.java
src/main/java/fr/inria/corese/benchmark/metrics/MetricsCollector.java
src/main/java/fr/inria/corese/benchmark/metrics/BenchmarkResult.java
src/main/java/fr/inria/corese/benchmark/Main.java
Update build.gradle for Java 21, remove Groovy