Table of Contents
It is possible to have more than one ClusterControl server to monitor a single cluster. This is useful if you have a multi-datacenter cluster and you may need to have ClusterControl on the remote site to monitor and manage the alive nodes if the connection between them goes down. However, ClusterControl servers must be configured to be working in active-passive mode to avoid race conditions when recovering failed nodes or clusters.
In the active mode, the ClusterControl node acts as a primary controller, where it performs automatic recovery and management activities. Therefore the primary controller Cluster/Node Auto Recovery must be set to on. The secondary ClusterControl node however must be configured with Cluster/Node Auto Recovery turned off.
Installing Standby Server
The steps described in this section must be performed on the secondary ClusterControl server.
- Install ClusterControl as explained on the Getting Started page.
- Import the same cluster via ClusterControl GUI → Deploy a cluster → Import a database cluster. Ensure to toggle off Cluster auto-recovery and Node auto-recovery in the Node configuration section. Repeat this step if you want to import more than one cluster.
- Set up the cluster configuration accordingly to follow similar settings with the primary ClusterControl (backup schedules, alerting configuration, user roles, etc).
Nothing should be performed on the primary side. The primary ClusterControl server shall perform automatic recovery in case of node or cluster failure. Use the secondary ClusterControl server for monitoring purposes only. For management and recovery purposes like rebuilding the replication, resyncing the node, backup, and restore, perform those activities on the primary ClusterControl server. These two ClusterControl servers are independent of each other.
Activating the secondary standby
If you want to make the standby server runs in the active mode, you must do the following:
- If the primary ClusterControl server is still alive, stop the primary ClusterControl controller services, or shut down the server. To stop all ClusterControl processes, run the following command on the primary ClusterControl server:
systemctl stop cmon cmon-cloud cmon-ssh cmon-events - Toggle on Cluster auto-recovery and Node auto-recovery on the secondary ClusterControl server.
At this point, the standby server has taken over the primary role and you can perform the management activities on the database nodes or clusters.
Attention
Do not let two or more ClusterControl instances perform automatic recovery or any management/automation tasks to the same cluster at a given time. Use the secondary ClusterControl server for monitoring purposes only, until the primary ClusterControl is down or stopped.