Why do changes to the Validator set require a one Session delay?

Question

This comment in the Session pallet states:

/// Move on to next session. Register new validator set and session keys. Changes to the
/// validator set have a session of delay to take effect. This allows for equivocation
/// punishment after a fork.

We are developing a custom Validator selection mechanic, but when we want to change the Validator set using the Session pallet, we must perform two session rotations in order to get the new set to take effect.

Are we missing something with respect to the implementation of the Session pallet that would make this necessity clearer? It would be good to get an ELI5 of the reason for the two-Session requirement.

Answer 1

This came out of necessity when implementing BABE. In BABE randomness from the VRFs is collected on-chain throughout a session (or epoch in BABE lingo), which is then used in future epochs. By delaying the session we make sure that randomness collected on session N is only used at N+2, and validators who want to start validating at N+2 must already commit to it at session N, without knowing what the randomness for that session will be.

That said, the session pallet is generic and ready to be used with other consensus protocols that don't have this requirement, so I think it's legitimate that it should allow for instant transitions. There is an open issue for this: https://github.com/paritytech/substrate/issues/8650, you should raise your voice there. :)