Skip to content

feat(graph): add production graph metric lifecycle and queries - #503

Draft
ajroetker wants to merge 127 commits into
mainfrom
feature/graph-metrics-core
Draft

feat(graph): add production graph metric lifecycle and queries#503
ajroetker wants to merge 127 commits into
mainfrom
feature/graph-metrics-core

Conversation

@ajroetker

@ajroetker ajroetker commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

What changed

  • adds durable degree, PageRank, eigenvector, and compatible HITS metric lifecycles over graph indexes
  • runs resumable coordinator and worker/worker-pool roles through std.Io and the database backend runtime, with durable owner leases, page leases, attempt fencing, retries, publish verification, cleanup, and restart recovery
  • exposes direct top-k graph metric reads, traversal projection/order/filtering, freshness controls, score reranking, status/profile details, and fail-closed hosted fan-in
  • adds internal maintenance operations, runtime status/telemetry, a user-facing maintenance command plus hidden supervisor child alias, and production-shaped process gates
  • extends OpenAPI and the Go, TypeScript, Python, and Zig client/contracts, including ergonomic direct graph-metric SDK request fields
  • ports the comprehensive PageRank design and operations guide from the combined graph work

Architecture and UX

  • production graph maintenance does not spawn raw OS threads; it uses backend_runtime.io_impl, std.Io futures, mutexes, conditions, and bounded runtime lanes
  • the process harness uses the same opaque API-kernel + httpx composition as production
  • the primary database backend remains LSM by default; graph metrics do not implicitly enable or depend on LMDB
  • graph JSON tests use structural lib/json assertions rather than serialization-order substrings
  • direct graph-metric, rerank, pruner, and mixed queries use the general read path so post-processing and admission behavior remain correct
  • publication is atomic and fail-closed: missing, malformed, or non-finite final ranks fail the durable page instead of publishing zero-valued or partial PageRank, eigenvector, or paired HITS generations

Validation

  • focused missing-final-rank PageRank recovery regression — 1/1 passed, no leaks
  • focused missing-paired-HITS-rank recovery regression — 1/1 passed, no leaks
  • zig build graph-metric-unit-test -fincremental --summary all — 74/74 passed, no leaks
  • zig build graph-metric-integration-test -fincremental --summary all — 94/94 passed, no leaks
  • zig build graph-metric-process-test -fincremental --summary all — 25/25 build steps; all promotion/failover proof floors met
  • graph metric command, supervisor, and smoke gates passed
  • zig build antfly-main-test -fincremental --summary all — 8/8 passed, no leaks
  • make zig-generated-check — OpenAPI/MCP, Snowball, SQL grammar, C ABI header, and TLS checks passed
  • Go SDK generation, tests, vet, and race detector passed
  • TypeScript SDK generation/typecheck passed; 155 tests passed, 1 skipped
  • Python SDK generation, Ruff, Pyright, and 23 tests passed
  • git diff --check

Notes

  • merged current main and regenerated all affected clients; the PR is conflict-free at the pushed head
  • kept as draft for review because this intentionally expands the original kernel-only PR into the comprehensive graph-metrics production slice

@ajroetker ajroetker changed the title feat(graph): add storage-independent PageRank kernel feat(graph): add production graph metric lifecycle and queries Aug 20, 2026
AJ Roetker added 24 commits August 20, 2026 15:10
…-core

# Conflicts:
#	go/pkg/sdk/oapi/client.gen.go
#	zig/build.zig
…-core

# Conflicts:
#	go/pkg/sdk/oapi/client.gen.go
#	go/pkg/sdk/types.go
#	py/packages/sdk/tests/test_client.py
#	ts/packages/sdk/test/types.test.ts
#	zig/pkg/antfly/src/api/indexes.zig
… reads

Persist current-only packed graph ordinals, adapt projection sharing to budgets, and deduplicate immutable query aliases before admission. Unify durable snapshot score reads with sorted physical keys and reusable batch slabs. Add ownership, integrity, allocation, budget regressions and reproducible benchmarks for both paths.
…-core

# Conflicts:
#	zig/pkg/antfly/src/runtime_error_abi.zig
Bound row, routing, and range planning before allocation; preserve row ordering with shared prefix-key sorting and block spans. Fold validated borrowed topology using checkpoint-local vector and target scratch. Add allocation-failure coverage, equivalence tests, and scoped benchmarks.
Elide later stateful adjacency producers; preserve sealed publication barriers. Use adaptive sparse ordinal projections, single-flight decoded page fills, and authenticated headroom-sized warm-start windows. Add recovery, saturation, memory-budget regressions and sparse projection benchmarks.
Seal generation/filter/partition-scoped topology outside numerical job lifetimes, atomically pin adopters, and reclaim obsolete owners and packing attempts in bounded maintenance pages. Cover reopen, concurrent producers, filter removal, idle pools, and reuse benchmarks.
Separate bounded fair topology preparation from numerical admission, canonicalize paired HITS failures, preserve terminal diagnostics, and account live serverless transport memory. Replace iterative string cursors with sealed ordinal coverage, expand recovery regressions, fix the TestDirectory CI migration, and record cursor benchmarks.
…-core

# Conflicts:
#	go/pkg/sdk/oapi/client.gen.go
…-core

# Conflicts:
#	zig/build.zig
#	zig/build_test_filters.zig
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant