Skip to content

fix: complete on-chain TLC reconciliation for terminal sweeps and uncommitted removes (#1611, #1612) - #1618

Open
jjyr wants to merge 2 commits into
nervosnetwork:developfrom
jjyr:fix/issue-1611-terminal-0xfe-settlement
Open

fix: complete on-chain TLC reconciliation for terminal sweeps and uncommitted removes (#1611, #1612)#1618
jjyr wants to merge 2 commits into
nervosnetwork:developfrom
jjyr:fix/issue-1611-terminal-0xfe-settlement

Conversation

@jjyr

@jjyr jjyr commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

Two fixes for on-chain TLC reconciliation on force-closed channels, addressing two sister issues reported against the exact-TLC settlement model introduced in #1583.

1. Terminal final-party sweep leaves pending TLCs unreconciled (#1611)

After #1583 the watchtower skips every final-party unlock (0xFE/0xFF). When such an unlock consumes the last commitment cell (no successor commitment output), the terminal sweep carries every still-pending TLC off-chain without recording an exact settlement proof. The channel then keeps ONCHAIN_SETTLEMENT_CONFIRMED forever, reconciliation never completes, and RemoveTlc is never propagated upstream.

Fix (watchtower): detect a terminal settlement (no successor commitment output) in process_watched_settlement_tx and, when a final-party unlock is observed on such a transaction, persist an exact no-preimage settlement (OnChainTlcSettlement { preimage: None }) for every remaining tracked TLC. Intermediate final-party unlocks (with a successor commitment cell) keep the previous behavior: pending TLCs stay tracked.

2. Uncommitted RemoteRemoved TLC is skipped during on-chain timeout reconciliation (#1612)

When a peer RemoveTlc arrives while the channel is shutting down with WAITING_COMMITMENT_CONFIRMATION, set_offered_tlc_removed marks the TLC RemoteRemoved but the remove commitment handshake never completes, so removed_confirmed_at stays None and the upstream RemoveTlc is never propagated. On-chain timeout reconciliation skipped such TLCs because collect_onchain_timeout_settled_tlcs filtered on removed_reason.is_none(), while get_expired_offered_tlcs already scopes to unconfirmed removes via removed_confirmed_at.is_none(). The channel then finalized settlement and stopped, leaving the upstream offered TLC permanently present.

Fix (fiber): drop the redundant removed_reason filter so uncommitted removed TLCs are collected and relayed upstream once their on-chain timeout settlement is observed. Guard the OriginPayer path in both the live-actor and no-live-actor reconciliation flows against re-marking an already removed TLC (set_offered_tlc_removed asserts Committed).

TDD

  • terminal_final_party_unlock_records_settled_without_preimage fails on e6cb7ac7 (0 settlement records) and passes with the watchtower fix; covers both 0xFE and 0xFF.
  • collect_timeout_settled_collects_uncommitted_removed fails on e6cb7ac7 (0 collected) and passes with the fiber fix.
  • collect_timeout_settled_skips_already_removed now pins the confirmed-remove case (removed_confirmed_at set).

Validation

  • cargo nextest run -p fnn -p fiber-bin --no-fail-fast — 1200 passed, 8 skipped
  • cargo fmt --all -- --check
  • cargo clippy --all-targets --all-features -p fnn -p fiber-bin -- -D warnings

Notes

A symmetric fulfill variant of #1612 (uncommitted removed TLC settled on-chain with a preimage is skipped by collect_onchain_fulfilled_tlcs via can_reconcile_onchain_fulfillment) is out of scope and can be followed up separately.

jjyr added 2 commits August 7, 2026 21:35
…ervosnetwork#1611)

After nervosnetwork#1583 the watchtower skips every final-party unlock (0xFE/0xFF),
so a terminal sweep that consumes the last commitment cell leaves every
still-pending TLC without an exact settlement record. The channel then
keeps ONCHAIN_SETTLEMENT_CONFIRMED forever, reconciliation never
completes, and RemoveTlc is not propagated upstream.

Detect a terminal settlement (no successor commitment output) in
process_watched_settlement_tx and, when a final-party unlock is observed
on such a transaction, persist an exact no-preimage settlement for every
remaining tracked TLC. Intermediate final-party unlocks (with a successor
commitment cell) keep the previous behavior: pending TLCs stay tracked.

TDD: terminal_final_party_unlock_records_settled_without_preimage fails
on e6cb7ac (0 settlement records) and passes with the fix.
…ervosnetwork#1612)

When a peer RemoveTlc arrives while the channel is shutting down with
WAITING_COMMITMENT_CONFIRMATION, set_offered_tlc_removed marks the TLC
RemoteRemoved but the remove commitment handshake never completes, so
removed_confirmed_at stays None and the upstream RemoveTlc is never
propagated. On-chain timeout reconciliation skipped such TLCs because
collect_onchain_timeout_settled_tlcs filtered on removed_reason.is_none(),
while get_expired_offered_tlcs already scopes to unconfirmed removes via
removed_confirmed_at.is_none(). The channel then finalized settlement and
stopped, leaving the upstream offered TLC permanently present.

Drop the redundant removed_reason filter so uncommitted removed TLCs are
collected and relayed upstream once their on-chain timeout settlement is
observed. Guard the OriginPayer path against re-marking an already removed
TLC (set_offered_tlc_removed asserts Committed).

TDD: collect_timeout_settled_collects_uncommitted_removed fails on
e6cb7ac (0 collected) and passes with the fix; the existing
collect_timeout_settled_skips_already_removed test now pins the confirmed
remove case (removed_confirmed_at set).
@jjyr
jjyr force-pushed the fix/issue-1611-terminal-0xfe-settlement branch from 431bb01 to bac1dc6 Compare August 7, 2026 13:35
@chainTe

chainTe commented Aug 8, 2026

Copy link
Copy Markdown

Security review (GPT-5.6 Codex) of
e6cb7ac7770b1798a1ad5dfb9a8f4ae5db52036f..bac1dc6863643fe2eebc42b7f8dc24e2ab7d45b7
found two low-severity crash-consistency gaps:

  1. A forwarded TLC can reach terminal downstream persistence before its
    live-upstream removal is durably acknowledged. A process failure in that
    interval can lose automatic replay and leave upstream value locked until
    manual recovery or force close.
  2. An origin-payer TLC can reach terminal channel persistence before the
    fire-and-forget payment failure event is durably applied, leaving stale
    Inflight payment/retry state after a crash.

Both paths require a narrow victim process-failure window that is not shown to
be peer-controlled. The robust fix is an acknowledged or persistent outbox
handoff before clearing terminal reconciliation state, with idempotent startup
replay.

Focused tests collect_timeout_settled_collects_uncommitted_removed and
terminal_final_party_unlock_records_settled_without_preimage passed with the
sqlite feature. Broad tests were skipped; CI is expected to cover them. No
deterministic crash-injection test was run.

@gpBlockchain

Copy link
Copy Markdown
Contributor

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants