Raft's Cluster Membership Changes

Cluster membership changes

Previously, it was assumed that the configuration of servers participating in the consensus algorithm is fixed. Sometimes, it is necessary to change the configuration to replace failed servers or change the replication levels of servers. However, taking the entire cluster offline to update configuration files is risky. The associated risk could be an operator error causing unavailability. To automate configuration changes and avoid these issues, the Raft consensus algorithm incorporates a mechanism for configuration changes.

Raft’s configuration change process must provide safety, preventing the possibility of two servers getting elected as leaders for the same term. However, directly converting all servers from the old configuration to the new one is risky since it risks dividing the cluster into two isolated majorities. As a result, switching all servers atomically at once is not possible.

The following illustration depicts an issue: If we switch directly from one configuration to another, different servers will make that switch randomly. There must be a moment when two majorities would follow two different cluster configurations, possibly having two different leaders from two different configurations.