Surfaced while adding smoke tests (PR pending).
Observed
In the seeded reference run (NUM_PACKETS=1000, MAX_HOPS=25, seed 42) the cognitive router delivers 238/1000 packets (23.8%); the remaining packets exhaust the hop cap, mostly bouncing between explored nodes. The dumb router delivers 1000/1000.
Likely cause
CognitiveNode.choose_next_hop uses UCB1 over neighbour arms, but the reward (REWARD_FACTOR / link_latency) is purely link-local: nothing penalises paths that loop or fail to reach the gateway, so exploration keeps revisiting dead-ends and the hop cap converts them to drops.
Acceptance ideas (pick one, small)
- Add a mild loop penalty (negative reward when revisiting the packet's previous nodes), or
- Track per-arm delivery success (route-level reward) instead of link latency only, or
- Cap exploration after convergence (epsilon decay) once all arms have been pulled.
The README now documents the trade-off; this issue tracks actually fixing it.
Surfaced while adding smoke tests (PR pending).
Observed
In the seeded reference run (
NUM_PACKETS=1000,MAX_HOPS=25, seed 42) the cognitive router delivers 238/1000 packets (23.8%); the remaining packets exhaust the hop cap, mostly bouncing between explored nodes. The dumb router delivers 1000/1000.Likely cause
CognitiveNode.choose_next_hopuses UCB1 over neighbour arms, but the reward (REWARD_FACTOR / link_latency) is purely link-local: nothing penalises paths that loop or fail to reach the gateway, so exploration keeps revisiting dead-ends and the hop cap converts them to drops.Acceptance ideas (pick one, small)
The README now documents the trade-off; this issue tracks actually fixing it.