Skip to content

Refactor Non-Validator and Epoch to have a transition callback - #521

Merged
samliok merged 9 commits into
mainfrom
on-index
Aug 26, 2026
Merged

Refactor Non-Validator and Epoch to have a transition callback#521
samliok merged 9 commits into
mainfrom
on-index

Conversation

@samliok

@samliok samliok commented Aug 18, 2026

Copy link
Copy Markdown
Collaborator

The main feature of this PR is the callback in the Epoch and NonValidator structs that signal to parent components a transition is required. This fixes the non-validator bug of restarting in-between epochs when it shouldn't be.

In the Epoch struct, when a sealing block is indexed we will now call the OnSealingBlockIndex which will notify the instance of an epoch change. Similarly, the NonValidator has a TransitionToValidator config that notifies the Instance when it should transition an epoch change.

The main benefit is now the non-validator can signal to the Instance when it is ready to transition to a validator. Before, since the Storage component was handling this change, the Storage component needed to re-call into the non-validator to confirm whether the sealing block actually meant a transition was needed. Since the non-validator has all the info, that logic should stay within the non-validator. This also helps avoiding deadlock since we don't need to be concerned about re-grabbing the non-validator lock.

This PR also simplifies and refactors some of the logic for transitioning an epoch.

  • No more nodeRole iota
  • removes some methods that i felt weren't adding much such as transitionEpochNonValidator and transitionEpochValidator
  • other small cleanups that were kind of hard to separate from the main changes


if block.SealingBlockInfo() != nil {
// are we the highest validator
highestEpoch, highestValidatorSet := n.epochs.highestEpoch()

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

highestEpoch == md.Seq {

So this basically means we have replicated the sealing block of the last epoch?

And the reason this works is that we replicate backwards?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If so, can we add some comments?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, if we have an epoch in the epochs map, that means we have verified it's accepted by enough validators. Therefore, if the block we are indexing the sealing block that is part of the highest epoch we should check if we need to transition.

will add a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this is good enough, on second thought.

If we start the non-validator we initialize this map with the last thing we have in the ledger and then if we commit a sealing block we will transition, won't we?

I think we should peek at the P-chain and compare the validator set and only if they're the same, then trigger it.

Thoughts?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should peek at the P-chain and compare the validator set and only if they're the same, then trigger it.

Hmm not sure this works either, because what if we are in the middle of a transition? and the highest pchain validator set is different than the current epoch. Maybe our validator is down, and it needs to catch up in order to sign off on the approval that will transition the epoch.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so i think as long as we do #530 we should be able to know for sure that we have the latest epoch for real, thoughts?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea i think this is sufficient

Comment thread simplex/epoch.go Outdated
Comment thread adapters.go Outdated
type EpochAwareStorage struct {
// InstanceStorage is a wrapper around Storage that skips indexing Telocks
// and delegates post-index handling to a caller-provided onIndex hook.
type InstanceStorage struct {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the rename? The name doesn't say anything about what it is doing, but just says now who is consuming it.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think EpochAware was no longer relevant, i renamed to CallbackStorage 🤷

Comment thread instance.go Outdated
if numBlocks == 0 {
return metadata.StateMachineBlock{}, 0, fmt.Errorf("no genesis block found in storage")
switch {
case i.nv != nil:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we be more explicit about what i.nv != nil and i.e != nilmean?

Comment thread instance.go
i.e.Stop()
// Wipe out the WALs from the config so we won't try to load them again
if garbageCollectWAL {
i.Config.WALs = nil

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't this a problem? How are we going to re-create the WAL now?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this is a problem, but i just copied the code from before. Probably should address the WAL issues in a separate PR.

I had this PR, maybe we can add it on to that

Comment thread instance.go Outdated
// On epoch change, garbage collect the WAL to remove all entries from previous epochs.
if err := i.wal.GarbageCollect(math.MaxUint64); err != nil {
i.Config.Logger.Error("Error garbage collecting epoch config on epoch change", zap.Error(err))
func GetHighestValidatorSet(platform PlatformChain) (common.Nodes, error) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is only used once, why is it needed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed

Comment thread nonvalidator/non_validator_test.go Outdated
tests := []struct {
name string
setup func(t *testing.T) (*testChain, []*messageInfo)
expectedCalls []transitionCall

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this a slice and not a single element?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread instance.go
select {
case i.epochChanges <- ec:
return
// The slot holds a stale epoch change: take it, keep the newer of the two and retry.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not strictly opposed to this, but - do we really need it now? Do we have a test that fails if this isn't done?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it can happen because we aren't acquiring the lock so we shouldn't deadlock. I still think we should have it and maybe I can log a warning so our test fails if it ever does?

@samliok
samliok merged commit 6b9a7f2 into main Aug 26, 2026
7 checks passed
@samliok
samliok deleted the on-index branch August 31, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Only start an epoch instance if has synced to the current epoch and is part of the latest validator set

2 participants