NodeFix — BlockDAG mainnet checkpoint recovery

Fixes for a BlockDAG (QNG / MeerDAG + MeerEVM) pool-stack node that crash-loops at the mainnet epoch-recovery checkpoint, block #11836362, and a related dashboard display bug that makes a healthy node look stalled.

These are community workarounds, not official BlockDAG releases. Use on infrastructure you operate. Prefer an upstream-corrected binary when available.

The problem

The shipped blockdag-node binaries (observed on stack-v1.0.2, stack-v1.1.0, and rebuilds of them) contain a hard-coded mainnet epoch-recovery checkpoint at block #11836362. The hash baked into the binary does not match the hash the live network actually produced at that height, so every node hits:

panic: invalid mainnet epoch recovery checkpoint: block #11836362
  hash 0xb74015ba3f455434330e1956e89c60c7dea7bd3a76fe0198e83730f289129c88,
  want 0x03358813d8c2b5fd917dc912872defcccfa1a16cff88fc5db75c44f82c0e7ad0

…and crash-loops. Sync cannot advance past #11836362, so the pool and any dashboards that read the node go dark.

Three independent observations confirmed the binary — not local data — is the problem:

• Two independently-synced nodes both derived the same 0xb74015… hash at that block (they agree with each other and with the live network).
• A third-party public node was already synced past #11836362 on that same chain.
• The 0x03358813… value the binary demands appears in no peer's chain.

The fix

scripts/apply-checkpoint-fix.sh swaps the hard-coded wrong hash for the hash the live network actually has at #11836362. Both are 64-character ASCII hex strings of equal length, so the edit is byte-for-byte in place and leaves the binary layout untouched. The script:

• refuses to run unless the wrong-hash string is present exactly once,
• writes a timestamped .bak before editing,
• verifies the result afterward.

# 1. See what a binary currently carries (read-only)
scripts/check-binary-checkpoint.sh /path/to/blockdag-node

# 2. Patch it (writes a .bak next to it)
scripts/apply-checkpoint-fix.sh /path/to/blockdag-node

Deploying the patched binary — important

In the docker pool-stack the node binary is baked into the image at build time, not read live from bin/. A plain docker stop/start keeps running the old baked binary — the swap silently has no effect. Patch the binary in the build context (bin/blockdag-node), then rebuild and recreate:

cd /path/to/stack
docker compose build node
docker compose up -d --force-recreate node

Verify the running container actually has the patch:

docker cp node:/usr/local/bin/blockdag-node /tmp/bn
scripts/check-binary-checkpoint.sh /tmp/bn      # expect: live-hash PRESENT

Validating safely before touching production

Test the patched binary against a copy of your chain data in a throwaway container (different ports, no keystore/network.key) and confirm it imports past #11836362 with no panic before deploying to the real node.

Also included

patches/netstats-rate-window.md — a dashboard bug where the "import rate" averages over the full ~1h history buffer, so downtime samples make a fast node read ~2 blk/s with a multi-day ETA. Fix uses a short recent window. Purely a display issue; verify true throughput from RPC (snippet in that doc).

Rollback

Every step leaves a .bak. To revert: restore the .bak binary (or your original vendor binary), then rebuild + --force-recreate the container.

Report upstream

This relaxes a consensus checkpoint to the network-majority value. Report the mismatch to BlockDAG maintainers so an official corrected binary can replace this workaround.