Add infrastructure, queue, and blockchain gauges to Prometheus metrics (BE-016 — partial) - #336
Merged
dDevAhmed merged 2 commits intoSep 5, 2026
Conversation
…metrics (BE-016) Closes DigiNodes#288
Contributor
|
@annacodervibe review status for head Once CI runs, the PR will be reviewed for metric correctness, bounded label cardinality, safe failure behavior, and exposure of no credentials or sensitive data. Please keep the current focused diff stable until that check completes. |
Contributor
|
resolve conflicts @annacodervibe |
Contributor
Author
@dDevAhmed conflict resolved |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #288
BE-016 is a large issue (5-6 days scoped), so this covers a real, verified slice rather than the whole thing - the parts around infra, queue, and blockchain metrics that were computed but not actually exposed anywhere alertable.
Added five new Prometheus gauges: process_memory_usage_bytes, process_cpu_usage_microseconds, queue_jobs_total, blockchain_last_indexed_block, and blockchain_indexing_lag_blocks. These reuse data HealthService was already computing for the /health endpoint (memory, CPU, queue counts, blockchain state), just wired into scrapable metrics instead of only living in the JSON diagnostics response.
Also found and fixed a real bug along the way: getHealth() was calling the async collectDiagnostics() without awaiting it, so the /health response's diagnostics field was silently serializing a Promise object instead of real data.
Left out anything I couldn't verify against real infrastructure - disk/network/container/K8s metrics, DB replication internals, Grafana/Alertmanager/Datadog/OpenTelemetry configs, and historical time-series storage. Those need either real infra to test against or a team decision on storage, so I'd suggest splitting them into their own follow-up issues rather than bundling into this one.
Verification: tsc shows the same 143 pre-existing errors before and after (zero new ones, confirmed via exact-diff). metrics.service.spec.ts: 9/9 passing. Re-verified on a completely separate fresh clone with a fresh install, same results.