Skip to content

Add Database Node

Adding a database node in ClusterControl is similar to adding a database node or adding a primary node. To simplify the terminology in this page, the name Add Node is simply adding a primary database node or denoting database node is simply a primary database node.

A primary node denotes that it is a database node ready for accepting read and writes. In contrast with Add Replica Node, the Add Database Node refers only to a primary database node. If you are looking to add a replica instead, see Add Replica Node.

Adding a database node is a common goal in ClusterControl for improving the resilience, read scalability, and availability of your database systems; especially for having a disaster recovery located on a separate region, collocation, or in different geography. ClusterControl simplifies this process significantly by eliminating the hassle and manual work.

Info

Adding a database node stepwise, and that you can only add one node in the course of action. You can run an iterative process by automating this with ClusterControl CLI.

Support Matrix

The following database cluster types, vendors and topology supports adding a database node:

Database Vendor Topology
Galera Cluster MariaDB, Percona Galera certification-based replication
MongoDB MongoDB, Percona, MongoDB Enterprise Replica set, sharded cluster
Redis Redis, Valkey Sentinel, cluster
Microsoft SQL Server for Linux Microsoft Standalone, Always On availability group
Elasticsearch Elastic Single-node cluster, high availability cluster

Prerequisites

When ading a node, it is very important to determine and prepare your host to have the requirements such as hardware, network, and user that is necessary to be accessed by ClusterControl and other database nodes.

There are prerequisites that need to be fulfilled prior to the deployment for all database clusters:

  • Make sure the target database nodes are running on a supported architecture platform and operating system. See Hardware and Operating System.
  • Passwordless SSH (SSH using key-based authentication) is configured from the ClusterControl node to all database nodes. See SSH Key-based Authentication.
  • Make sure that the OS user you setup for passwordless login is the same user your ClusterControl and its cluster used during deployment.
  • Verify that sudo is working properly if you are using a non-root user. See Operating System User.

MySQL Galera

The following vendors and versions are supported for adding database node:

  • Percona XtraDB Cluster - 8.0.
  • MariaDB Galera - 10.4, 10.5, 10.6 (LTS), 10.11 (LTS) and 11.4 (LTS).

Adding a database node requires that your existing cluster must have at least one node running. When the job is in progress, ClusterControl will rely on the Galera's mechanism to do a full SST from source to the target node (database node to be setup and added). This means that ClusterControl relies on binary backup to stream data to copy from source to the target database. Binary backup does not lock your primary node once a new node is added. However, If you are concerned of the traffic and/or the data to be copied from your Primary node is big, we recommended to add it during low peak period or use the backup created taken using ClusterControl. This shall be discussed on how to rebuild from backup in the following section below.

Below is how ClusterControl relies on Galera's mechanism to apply a full SST:

  • For Percona XtraDB Cluster, it will use Percona XtraBackup to stream data from its existing primary to the target node.
  • For MariaDB Server, it will use Mariabackup to stream data from its its existing primary to the target node.

This is set through wsrep_sst_method upon deployment.

Default configuration

By default, ClusterControl deploys MySQL/MariaDB database with the following configurations:

  • During the job process, ClusterControl disables cluster and node recovery. It will re-instate its former values accordingly.
  • ClusterControl sets wsrep_sst_method to xtrabackup for Perocna XtraDB Cluster or mariabackup for MariaDB Galera.
  • MySQL GTID with log_slave_updates enabled (Oracle MySQL and Percona only).
  • MariaDB GTID with log_slave_updates enabled (MariaDB only).
  • a database node shall act as a primary having read_only=OFF and super_read_only=OFF (if supported).
  • performance_schema is enabled.
  • The generated account credentials are stored inside secrets-backup.cnf under the MySQL configuration directory.

Note

If you would like to customize the above configurations, modify the template base file to suit your needs before proceeding to the deployment. See Configuration Template for details.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add new → Primary node

  3. Choose Create a primary node

  4. First step is Node configuration.

    1. Data directory is set to /var/lib/mysql. If you have prepared a separate path or on a different volume, replace the value with your custom path.

    2. Galera Segment is set to 0 by default. If you are adding a node into different segment, set the value here i.e. > 0. See here for more about segments in Galera.

    3. Configuration template is set to my.cnf.[galera-version]galera where [galera-version] entails the vendor and version of Galera ClusterControl intends to add this node. If you have your own custom template, ClusterControl will show the templates that has been added or configured in /etc/cmon/templates. For more details see Configuration template

    4. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package version instead.

    5. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  5. Click Continue.

  6. Second step is Advanced Settings.

    1. Rebuild from a backup is disabled. If you desire to rebuild your data instead of streaming from the Primary node, toggle it to On. This shall reveal a drop-down list with all full-backup created using Clustercontrol.

    2. Include in LoadBalancer set (if exists) is disabled. Switch it to *On if you want ClusterControl to discover any existence of load balancer that ClusterControl supports.

  7. Click Continue.

  8. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  9. Click Continue to proceed the last step Preview.

  10. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  11. Click Finish button to deploy the job.

To be added soon

Recommended Next Steps

MongoDB Replica Set

The following vendors and versions are supported for adding database node:

  • MongoDB Enterprise - 4.4, 5.0, 6.0 and 7.0
  • MongoDB Community - 4.4, 5.0, 6.0 and 7.0
  • Percona Server for MongoDB - 4.4, 5.0, 6.0 and 7.0

Adding a database node requires that your existing cluster must have at least one node running as primary. If no primary found, ClusterControl will eventually fail the job process since it will fail to identify an elibible primary in that replica set.

When the job is in progress, ClusterControl will execute the rs.add() function to add the new node to its target replica set. If you are concerned of the traffic and/or the data to be copied from your Primary node is big, we recommended to add it during low peak period.

Once the adding a node job is done, the new node shall have Role of Secondary once viewed in the Nodes tab.

Default configuration

By default, ClusterControl deploys MongoDB Replica Set members with the following configurations:

  • Configure setParameter.enableLocalhostAuthBypass: true inside MongoDB configuration file.
  • ClusterControl will create and grant necessary roles for additional MongoDB users - admin.cmon_backup for backup and restore purposes and admin.cmonexporter for query monitoring.
  • Configure sharding.clusterRole: shardsvr inside MongoDB configuration file.

If you would like to customize the above configurations, modify the template base file to suit your needs before proceeding to the deployment. See Configuration Template for details.

Attention

It is possible to deploy only a DB server only that has one primary database node already in the replica set and no other nodes or arbiter. The caveat of this approach is no automatic failover. If the primary node goes down then manual failover is required to make the remaining secondary node run as primary. Automatic failover works fine with 3 nodes and more.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add new → Primary node

  3. Choose Create a primary node

  4. First step is Node configuration.

    1. Node type has values Db server and Arbiter. Choose Db server to add a primary database node.

    2. Replica set is a drop-down list. When click this shall reveal replica set created in the cluster. Make sure you know the replica set you are intending this node to be assigned and choose it as you desire.

    3. Specify the Port of your target replica node. Default is set to 27017.

    4. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package and version of the database instead.

    5. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  5. Click Continue.

  6. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  7. Click Continue to proceed the last step Preview.

  8. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  9. Click Finish button to deploy the job.

To be added soon

MongoDB Sharded Cluster

The following vendors and versions are supported for adding database node:

  • MongoDB - 4.4, 5.0, 6.0 and 7.0
  • MongoDB Enterprise - 4.4., 5.0, 6.0 and 7.0
  • Percona - 4.4, 5.0, 6.0 and 7.0

Adding a database node requires that your existing MongoDB Sharded Cluster must have at least one node running as primary. If no primary found, ClusterControl will eventually fail the job process since it will fail to identify an eligible primary in that replica set. Whether this replica set is a Config Server or Shard Server, it requires a primary node to run healthy so that the new node can join the cluster.

When the job is in progress, ClusterControl will execute the rs.add() function to add the new node to its target replica set. If you are concerned of the traffic and/or the data to be copied from your Primary node is big, we recommended to add it during low peak period. If you are going to add a router or mongos, this can be quick since router will act as your load balancer and does not store data. ClusterControl can configure quickly and does not have any impact at all to your database nodes or existing MongoDB cluster.

Once the adding a node job is done, the following roles shall be assigned:

  • Config Server. This role is assigned if you added a Config server node
  • Mongos. This role is assigned if you added a router/mongos node. This is your load balancer.
  • Primary/Secondary. If you have added the node as part of your sharded server, then primarily, the new node shall have Secondary since your existing cluster shall have an assigned primary role assigned.

Default configuration

By default, ClusterControl deploys MongoDB Sharded Cluster with the following configurations:

  • Configure setParameter.enableLocalhostAuthBypass: true inside MongoDB configuration file.
  • ClusterControl will create and grant necessary roles for additional MongoDB users - admin.cmon_backup for backup and restore purposes and admin.cmonexporter for query monitoring.
  • Configure sharding.clusterRole: shardsvr inside MongoDB configuration file if node added is a DB server and part of the shard.
  • Configure sharding.clusterRole: configsvr inside MongoDB configuration file if node added is a Configuration server.
  • sets the sharding.configDB to the replica set config servers when adding a router/mongos.
  • sets security.keyFile: /etc/mongo-cluster.key when adding a router/mongos.

Attention

It is possible to deploy nodes either router, config, or database node with less than 3 nodes. However, for high availability and no single-poit of failure (SPOF), we recommend to make sure to have at least 2 or 3 nodes per shard, at least 2 nods for config server, and 2 nods for your router/mongos.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add new → Node

  3. Choose Create a database node

  4. First step is Node configuration.

    1. Create router/mongos. By default, this is Off.

    2. Node type has values Db server, Config server, and Arbiter. Choose Db server to add a primary database node with shardsvr role. Choose Config server if you add it as a configsvr role, or arbiter as the name suggest.

    3. Replica set is a drop-down list. When click this shall reveal replica set created in the cluster. Make sure you know the replica set you are intending this node to be assigned and choose it as you desire.

    4. Specify the Port of your target replica node. Default is set to 27017. Make sure that you set the port correctly to which port you are aiming or to make it uniform as what your existing node roles are using, but not required instead it's recommended.

    5. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package and version of the database instead.

    6. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  5. Click Continue.

  6. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  7. Click Continue to proceed the last step Preview.

  8. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  9. Click Finish button to deploy the job.

To be added soon

Redis Sentinel

The following vendors and versions are supported for adding database node:

  • Redis Sentinel - 6 and 7

Essentially, as mentioned earlier, adding a database node in this page is simply adding a primary database node. However, in this case, action goes for adding a node but the end result, the newly database node added shall act as a replica, instead of a primary. In this manner, adding a database node requires a primary node in the course of action in order to complete the job successfully.

If the existing primary is down, job can still be done successfully. However, the primary node needs to be up sometime so that the replica can send a PSYNC to the primary node when its up and it can start syncing its data from the primary going to be replicated down to the replica node (the node that had just been added).

Apart from adding your database node as secondary, ClusterControl will also deploy Sentinel node which shall run on port 26379.

Default Configuration

By default, ClusterControl deploys Redis cluster instances with the following configurations:

  • Default port points to 6379 for the Redis server and port 26379 for Sentinel service.
  • ClusterControl will configure the Redis instances with appendonly enabled.
  • ClusterControl will secure the instance with AUTH enabled and configure the requirepass and masterauth options.
  • ClusterControl will enable TLS encryption. To access the cluster, one must use the --tls flag to connect.
  • The configuration maxMemory (70% of node's RAM, rounded to the nearest power of 2) and maxMemoryPolicy=allkeys-lru will be set, to reduce the risk of Redis being killed by OOM.
  • Deploys sentinel node co-located within the same node (Redis instance) being added.

Tips

Redis Sentinel requires 3 nodes for automatic primary promotion. When you have only one-single Redis Sentinel node, make sure you add two or make sure to have an odd total number of nodes per cluster.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add node

  3. First step is Node configuration.

    1. Specify the Port of your target database (replica) node. Default is set to 6379.

    2. Specify the Redis sentinel port for your sentinel service. Default is set to 26379.

    3. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package and version of the database instead.

    4. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  4. Click Continue.

  5. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  6. Click Continue to proceed the last step Preview.

  7. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  8. Click Finish button to deploy the job.

To be added soon

Redis/Valkey Cluster

The following vendors and versions are supported for adding a database node:

  • Redis Cluster - 6 and 7
  • Valkey Cluster - 7 and 8 (only RHEL 8/9 are supported)

Adding a database node (primary database node) requires a running and healthy primary node in the course of action in order to complete the job successfully. When the job is in progress, ClusterControl will setup the database node then executes CLUSTER MEET and rebalances the slots in the cluster to ensure a more even distribution across all nodes.

Default Configuration

By default, ClusterControl deploys Redis cluster instances with the following configurations:

  • ClusterControl will configure the Redis/Valkey instances with appendonly enabled.
  • ClusterControl will secure the instance with AUTH enabled and configure the requirepass and masterauth options.
  • ClusterControl will enable TLS encryption. To access the cluster, one must use the --tls flag to connect.
  • The configuration maxMemory (70% of node's RAM, rounded to the nearest power of 2) and maxMemoryPolicy=allkeys-lru will be set, to reduce the risk of Redis being killed by OOM.
  • ClusterControl uses cluster bus port i.e. main-port + 1000 unless manually specified during cluster deployment.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add new → Primary Node

  3. First step is Node configuration.

    1. Specify the Port of your target database node. Default is set to 6379.

    2. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package and version of the database instead.

    3. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  4. Click Continue.

  5. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  6. Click Continue to proceed the last step Preview.

  7. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  8. Click Finish button to deploy the job.

To be added soon

SQL Server

The following versions for Microsoft SQL Server are supported for adding a database node:

  • Microsoft SQL Server for Linux - 2019 and 2022

Adding a database node will deploy a secondary database node. In this action, ClusterControl will setup the node as part of the Always On Availability Groups (AGs). This requires to have a running and healthy primary node in the course of action in order to complete the job successfully.

Attention

Only hostname or FQDN is supported. Therefore, proper host naming and mapping must be performed beforehand. You may use /etc/hosts or DNS mapping to achieve this. When adding a database node in ClusterControl, entering an IP address will produce an error.

License

Deploying SQL Server with ClusterControl does not change how you must license SQL Server. Each node where SQL Server is running requires appropriate licensing, just as it would if you deployed SQL Server manually. The presence of ClusterControl as an orchestration or management layer has no bearing on Microsoft’s licensing rules.

When deploying SQL Server in a clustered environment, licensing rules still apply to each server running SQL Server. However, the specifics differ depending on whether the node is actively running SQL Server or is a passive failover node:

  • Active Nodes: Each active node that runs SQL Server services needs to be fully licensed.
  • Passive Failover Nodes: Under current SQL Server licensing rules, you can typically have one passive failover node per licensed instance without needing an additional license for that passive node. The passive node is considered a “standby” server that can take over if the active server fails. It must truly remain passive, running no active workloads except those required for maintaining the standby state.

Obtain your SQL Server license directly from Microsoft, a certified Microsoft Partner, or a cloud marketplace (if applicable).

Default configuration

By default, ClusterControl deploys SQL Server with the following configurations:

  • All database nodes must use fully qualified domain name (FQDN). IP address is not supported.
  • Enforces the SQL Server user's password policy as shown here.
  • At the moment, ClusterControl only deploys AlwaysOn with asynchronous-commit mode, where it does not wait for any secondary replica to write incoming transaction log records to disk.
  • Sets hadrenabled = 1 in /var/opt/mssql/mssql.conf config.
  • Enables sqlagent in /var/opt/mssql/mssql.conf config.

Deployment steps

  1. Go to the Clusters dashboard. Choose your target source cluster and click the ellipsis button (...)

  2. Choose Add node

  3. First step is Node configuration.

    1. Port of your target database node. It is set to 1433 and currently, this is disabled.

    2. Install software is set to On. You can disable this if you do not want to install the software if the package is already installed. If set to On, ClusterControl will uninstall any existing package found and install the right package and version of the database instead.

    3. Security configuration to either disable firewall or SELinux/AppArmor. Leave check if you need to Disable firewall (default). Leave check if you need to Disable SELinux/AppArmor (default). Otherwise uncheck if any of the two needs not be disabled. If you have uncheck Disable SELinux/AppArmor, make sure that you have configured properly to allow creating files and opening sockets for the database to run properly.

  4. Click Continue.

  5. Third step is Add node.

    1. In the Node field, this is where you have to specify and input the IP address or hostname of your target node. If you intend to add another one, this will just overwrite what was previously inputted and added since it only allows one node at a time.
  6. Click Continue to proceed the last step Preview.

  7. Fourth and the last step is Preview, this is the last step where it shows all the values and choosen options for the ClusterControl to use for setting up the node.

  8. Click Finish button to deploy the job.

To be added soon