Cluster Management

This feature provides information and management options for database clusters managed by ClusterControl. Clicking on the Clusters (sidebar menu) will list out all database clusters on the main panel, each of the entries has a real-time summary of the cluster, together with the dropdown management menu (see Cluster Actions):

Each entry will have the following details:

Cluster name: Configurable during import or deployment, also via the s9s command line.
Cluster ID: Immutable ID assigned by ClusterControl after it is registered.
Cluster type: The database cluster type, recognized by ClusterControl. See Supported Databases.
Nodes: All nodes grouped under this particular cluster.
Auto recovery: The ClusterControl automatic recovery settings. See Failover and Recovery.
Load: Last 5 minutes of cluster load average.
...: Every supported database cluster has its own set of dropdown menus. See Cluster Actions.

Choose a database cluster from the expandable list on the left-side menu to drill down on the cluster-specific features. The navigation breadcrumbs (top panel) will reflect with the corresponding cluster when selecting a cluster.

Cluster Actions

Provides shortcuts to the main cluster functionality. Each database cluster has its own set of functionality as described below:

MySQL/MariaDB/Percona (replication/standalone)

Feature	Description
Add new → Replication node	Adds a new or existing replica database node into the cluster. The new replica will automatically join and synchronize with the rest of the cluster. See Add Replication Node.
Add new → Load balancer	Adds a new or existing load balancer into the cluster. See Create Load Balancer.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Encryption settings → Change SSL Certificate	Opens the SSL certificate management wizard which allows you to create or choose an existing certificate for database client-server encryption
Encryption settings → SSL encryption	Toggles On to enable client-server SSL encryption. This action will restart the whole cluster, one node at a time Toggles Off to disable client-server SSL encryption. This action will restart the whole cluster, one node at a time
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Audit logging	Enable audit logging (MariaDB-based clusters only) using policy-based monitoring and logging of connection and query activity. This feature will enable audit logging on all nodes in the cluster. See Audit Logging.
Readonly	Toggles On to enable cluster-wide read-only. ClusterControl will configure `read_only=ON` and `super_read_only=ON` (except MariaDB) on all database nodes in the cluster Toggles Off to disable cluster-wide read-only
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, for example from 8.0.x to 8.0.y, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades
Create database	Create a new database schema on the primary node. For MySQL, the database name can contain letters, numbers, `$` and `_` symbols. Maximum length is 64 characters.
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration

Galera Cluster (Percona XtraDB/MariaDB)

Feature	Description
Add new → Node	Adds a new or existing database node into the cluster. You can scale out your cluster by adding more database nodes. The new node will automatically join and synchronize with the rest of the cluster. See Add Node.
Add new → Replication node	Adds a new or existing replica database node into the cluster. The new replica will automatically join and synchronize with the rest of the cluster. See Add Replication Node.
Add new → Load balancer	Adds a new or existing load balancer into the cluster. See Create Load Balancer.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Encryption settings → Change SSL Certificate	Opens the SSL certificate management wizard which allows you to create or choose an existing certificate for database client-server encryption
Encryption settings → Change Galera SSL Certificate	Opens the SSL certificate management wizard which allows you to create or choose an existing certificate for Galera cluster nodes communication encryption
Encryption settings → Galera SSL encryption	Toggles On to enable Galera SSL encryption. This action will restart the whole cluster, one node at a time Toggles Off to disable Galera SSL encryption. This action will restart the whole cluster, one node at a time
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Readonly	Toggles On to enable cluster-wide read-only. ClusterControl will configure `read_only=ON` and `super_read_only=ON` (except MariaDB) on all database nodes in the cluster Toggles Off to disable cluster-wide read-only
Clone/Replicate the cluster → Clone cluster	This feature has been deprecated. Use Create cluster from backup instead.
Clone/Replicate the cluster → Create replica cluster	Create a new cluster that replicates this cluster. See Create Replica Cluster.
Find most advanced node	Finds which is the most advanced node in the cluster. This is useful to determine which node to bootstrap if the cluster doesn’t have any primary component or when ClusterControl automatic recovery is disabled.
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, for example from 8.0.x to 8.0.y, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades
Create database	Create a new database schema on the primary node. For MySQL, the database name can contain letters, numbers, `$` and `_` symbols. Maximum length is 64 characters.
Rolling restart	Performs a cluster restart operation one node at a time. The rolling restart will be aborted if a node fails to be restarted For the Galera Cluster, toggle On *Perform SST* to force a full snapshot state transfer. The data directory will be removed from the cluster and the node will be completely resynced from another node. Note that this usually takes a longer time to complete depending on the dataset size It is also possible to reboot the host one at a time by toggling the *Reboot hosts* button.
Bootstrap cluster	Launches the bootstrap cluster window. ClusterControl will stop all running nodes before bootstrapping the cluster from the selected Galera node. See Bootstrap Cluster.
Stop Cluster	Stop all nodes in the cluster. This is the recommended way to perform a graceful shutdown to the cluster.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration.

PostgreSQL/TimescaleDB (streaming replication/standalone)

Feature	Description
Add new → Replication node	Adds a new or existing replica database node into the cluster. The new replica will automatically join and synchronize with the rest of the cluster. See Add Replication Node.
Add new → Load balancer	Adds a new or existing load balancer into the cluster. See Create Load Balancer.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Encryption settings → Change SSL Certificate	Opens the SSL certificate management wizard which allows you to create or choose an existing certificate for database client-server encryption
Encryption settings → SSL encryption	Toggles On to enable client-server SSL encryption. This action will restart the whole cluster, one node at a time Toggles Off to disable client-server SSL encryption. This action will restart the whole cluster, one node at a time
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Audit logging	Enable audit logging using policy-based monitoring and logging of connection and query activity. This feature will enable audit logging on all nodes in the cluster. See Audit Logging.
Create replica cluster	Create a new cluster that replicates this cluster. See Create Replica Cluster.
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, for example from 8.0.x to 8.0.y, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades
Enable TimescaleDB	Installs TimescaleDB database extension on every PostgreSQL node. The installation will be performed on one database node at a time, followed by a database restart to apply the changes Once the PostgreSQL is converted into TimescaleDB, this action will not be reversible. ClusterControl will treat the cluster as a TimescaleDB cluster onwards
Create database	Create a new database schema on the primary node. For PostgreSQL, the database name must start with a letter or underscore, and can contain only letters, numbers and underscores. Maximum length is 63 characters.
Edit details	Edit the cluster’s name and tags.
Setup logical replication	Set up logical replication between two existing PostgreSQL clusters using a publish-and-subscribe model. This is only supported for PostgreSQL 16 and later. Choose an existing PostgreSQL cluster from the Target cluster dropdown. The target cluster should not be part of another logical replication. This operation establishes a new logical cluster to manage publications and subscriptions. To enable bi-directional replication, configure both clusters with publications and subscriptions. See PostgreSQL Logical Replication.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration.

PostgreSQL Logical Replication

Feature	Description
Actions → Edit details	Edit the cluster’s name and tags.
Actions → Remove cluster	Remove the logical replication grouping for all clusters connected to it. The PostgreSQL database nodes or streaming replication cluster will be ungrouped from the logical binding and will continue to operate as a normal standalone node or streaming replication cluster After the removal, the WAL level will remain as logical, which is usable for streaming replication as well (despite the WAL size will be bigger if compared to replica or hot_standby) One must explicitly type “REMOVE” in the text field before triggering the cluster removal job
	Pubs & Subs → Create publication
Publisher cluster	Choose a database cluster from the dropdown list as the publisher cluster.
Database	Choose a database from the list. The target database and its corresponding objects like tables must be created first in the publisher cluster. This list will be auto-populated by ClusterControl depending on the *Publisher cluster*
Publication name	Specify a publication name
Include all tables	Toggle on to include all tables under the chosen database Toggle off will open a dropdown editor where you can type the table name and press `Enter`. Multiple values are supported
	Pubs & Subs → Create subscription
Subscriber cluster	Choose a database cluster from the dropdown list as the subscriber cluster
Subscription name	Specify the subscription name
Publication	Choose an existing publication from the list
Database	Choose a database from the list. The target database and its corresponding objects like tables must be created first in the subscriber cluster. This list will be auto-populated by ClusterControl depending on the Subscriber cluster
copy data	Toggle on to copy pre-existing data in the publications that are being subscribed to when the replication starts Toggle off will skip the initial snapshot stage. It is recommended for the user to manually perform the initial syncing before proceeding with this option
Origin	None means the subscription will only replicate changes that do not have an associated origin (i.e., changes made locally), while Any means the subscription will replicate all changes regardless of their origin, including those already replicated from other nodes This operation establishes a new logical cluster to manage publications and subscriptions. To enable bi-directional replication, configure both clusters with publications and subscriptions. See PostgreSQL Logical Replication
	Alarms
Actions → Details	Open the alarm details page presenting the alarm status, description, and recommendation
Actions → Copy link	Copy the individual direct link to the alarm details
Actions → Mute	Mute the alarm. Once muted, the individual direct link to the alarm details
	Jobs
Actions → Details	Open the job details page presenting the job status, cluster information, job specifications, and job messages Click on Copy to clipboard to copy the whole information in this page into the clipboard. Useful for information sharing and troubleshooting.
Actions → Copy link	Copy the individual direct link to the job details and messages.
Actions → Delete	Delete the job from ClusterControl records for this cluster

Attention

For PostgreSQL logical replication, setting origin=any can lead to a potential infinite loop in bi-directional replication setups, as changes replicated from one node can be sent back to the originating node. To avoid this, users should carefully configure subscriptions and use origin=none to prevent such behavior.

MongoDB

Feature	Description
Add node	Scale the current MongoDB Replica Set or Sharded Cluster deployment by adding a single shard, Mongos, or config server.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Convert to shard	Only for MongoDB Replica Set. Converts an existing MongoDB replica set to a sharded cluster by adding Mongos and config servers into the setup
SSL encryption	Toggles On to enable client-server SSL encryption. This action will restart the whole cluster, one node at a time Toggles Off to disable client-server SSL encryption. This action will restart the whole cluster, one node at a time
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration

Redis

Feature	Description
Add node	Adds a new or existing node into the cluster. You can scale out your cluster by adding more database nodes. The new node will automatically join and synchronize with the rest of the cluster. See Add Node.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, for example from 7.0.x to 7.0.y, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration

Microsoft SQL Server

Feature	Description
Add node	Adds a new or existing node into the cluster. You can scale out your cluster by adding more database nodes. The new node will automatically join and synchronize with the rest of the cluster. See Add Node.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	This feature is not available yet for MSSQL-based clusters
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration

Elasticsearch

Feature	Description
Add node	Adds a new Elasticsearch master, data, coordinator, or master-data node into the cluster. The new master or data node will automatically join and synchronize with the rest of the cluster. See Add Node.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	This feature is not available yet for Elasticsearch-based clusters
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration

Valkey

Feature	Description
Add node	Adds a new or existing node into the cluster. You can scale out your cluster by adding more database nodes. The new node will automatically join and synchronize with the rest of the cluster. See Add Node.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	Performs minor software upgrades for the database and load balancer software of this cluster, for example from 7.0.x to 7.0.y, in a rolling upgrade fashion. ClusterControl will perform the software upgrade based on what is available on the package repository for the particular vendor. See Upgrades.
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal. One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration.

ClickHouse

Feature	Description
Add new → Data node	Adds a new ClickHouse data node to the cluster, either as a new replica in an existing shard or as a new shard. See Add Database Node → ClickHouse.
Add new → Keeper node	Adds a new node to the ClickHouse Keeper ensemble, growing it for quorum. See Add Database Node → ClickHouse.
Schedule maintenance	Schedules a cluster-wide maintenance mode, where ClusterControl will skip raising alarms and notifications while the mode is active All nodes in the cluster (regardless of the role) will be marked as under maintenance. A global banner will appear if there is upcoming maintenance for the corresponding cluster
Cluster/node recovery → Cluster recovery	Disables or enables cluster recovery If enabled, ClusterControl will attempt to recover the cluster if necessary. See Cluster Recovery.
Cluster/node recovery → Node recovery	Disables or enables node recovery If enabled, ClusterControl will attempt to recover the node if necessary. See Node Recovery.
Change RPC API token	Serves as the authentication string by ClusterControl UI to connect to the CMON RPC interface. Each cluster has its unique token
Upgrades	Performs software upgrades for the ClickHouse cluster in a rolling upgrade fashion, based on what is available on the package repository. See Upgrades.
Edit details	Edit the cluster’s name and tags.
Remove cluster	Removing the cluster will delete all metadata we have on it. The database nodes will not be affected and will continue to operate as normal. One has to explicitly type “REMOVE” in the text field before the cluster removal job is triggered. The cluster can be imported again in the future with fresh new metadata and configuration.

Not supported for ClickHouse

The following actions, available for other cluster types, are not available for ClickHouse:

Readonly (cluster-wide read-only toggle): ClickHouse does not have a MySQL-style cluster-wide read-only mode.
Create database: create databases directly on the ClickHouse cluster instead.
Encryption settings → Change/Enable SSL Certificate: ClusterControl deploys ClickHouse with SSL/TLS enforced by default (plaintext ports are disabled), so there is no separate toggle to turn it on or off.

Cluster-specific actions

Bootstrap cluster

Bootstrap cluster refers to the process of bootstrapping or initializing a new Galera Cluster. When you’re setting up a new Galera Cluster or performing a full backup restoration, you typically need to bootstrap it, which involves designating one node as the initial primary component of the cluster. This bootstrap node serves as the starting point for the cluster, and other nodes will then join this initial node to form the cluster.

The following actions will be performed:

The cluster will be initialized from the selected node.
All nodes will be stopped unless they are already stopped.
When the bootstrap command is successful, the selected node will be Synced. The rest of the nodes will be started as joiners, one node at a time.

Field	Description
Bootstrap node	Choose a database node from the list to be the reference node. If you are unsure, you may choose Auto Select from the dropdown where ClusterControl will determine the most suitable node to bootstrap based on the Galera node’s sequence number.
Graceful shutdown timeout (in seconds)	ClusterControl will wait for this timeout to finish before forcing the node to stop.
Force stop the nodes (after shutdown time)	Toggles On to force stop the database node if it takes longer than the shutdown time, specified in Graceful shutdown timeout (in seconds), defaulting to 1800 seconds (30 minutes). Commonly useful in the case of forcing a database cluster to bootstrap when database locking or long-running transactions are happening.
Clear MySQL datadir on Joining nodes	Toggles On to clear the MySQL data directory on the joining nodes. This is often necessary after a restoration.

How ClusterControl Manages MSSQL on Linux

This section explains how ClusterControl deploys and manages Microsoft SQL Server on Linux. It covers the supported licenses, the differences between a standalone instance and an Always On availability group, the configuration that ClusterControl applies during deployment, and the current limitations. For the deployment steps, see Create Database Cluster.

Supported SQL Server licenses

ClusterControl installs SQL Server using mssql-conf setup and passes the chosen edition as MSSQL_PID. The following editions are accepted:

Edition	Description
Evaluation	The default edition. ClusterControl uses it when no license is specified during deployment. Includes all Enterprise features, free for 180 days.
Developer	All Enterprise features, but licensed for development and testing only. Not for production use.
Express	Free edition with resource limits set by Microsoft. It does not support Always On availability groups, so use it for standalone instances only.
Web	For public web hosting under a Microsoft SPLA agreement. It does not support Always On availability groups.
Standard	Paid edition. Microsoft limits Standard to basic availability groups. When the license is Standard, ClusterControl configures the secondary replicas as non-readable (`ALLOW_CONNECTIONS = NO`).
Enterprise	Paid edition with the full Always On feature set, including readable secondary replicas.

To deploy with an edition other than Evaluation, use the ClusterControl CLI with the --license option. The value is not case-sensitive:

s9s cluster --create \
    --cluster-type=mssql_ao_async \
    --vendor=microsoft \
    --provider-version=2022 \
    --license=enterprise \
    --db-admin=SQLServerAdmin \
    --db-admin-pass='NqR$feGR68xli^#v' \
    --nodes="mssql://dsa-mssql-db1:1433;dsa-mssql-db2:1433;dsa-mssql-db3:1433;" \
    --os-user=root \
    --os-key-file=/root/.ssh/id_rsa \
    --cluster-name="MSSQL Server 2022 Always On" \
    --log

The edition you choose is saved in the cluster configuration. Every node added later is installed with the same edition.

Note

ClusterControl does not accept a SQL Server product key during deployment. If you have purchased a license, deploy with the matching edition name (for example standard or enterprise), or deploy with Evaluation and switch the edition on each node afterwards using Microsoft's mssql-conf set-edition command:

sudo systemctl stop mssql-server
sudo /opt/mssql/bin/mssql-conf set-edition
sudo systemctl start mssql-server

ClusterControl does not detect an edition change made outside of ClusterControl. If you change the edition, nodes you add later are still installed with the edition stored in the cluster configuration.

Using ClusterControl to deploy or manage SQL Server does not change how Microsoft licenses SQL Server. Each node running SQL Server must be licensed as if you had set it up manually. Obtain your license from Microsoft, a Microsoft partner, or your cloud marketplace.

What happens when the license limit is exceeded

ClusterControl does not check or enforce SQL Server edition limits such as CPU cores, memory, database size, or the number of replicas in a basic availability group. These limits are enforced by SQL Server itself:

When the 180-day Evaluation period ends, SQL Server stops running and the service does not start again until the edition is changed to a licensed one. ClusterControl reports the affected nodes as failed. To recover, change the edition with mssql-conf set-edition as shown above.
If a job asks SQL Server to do something the edition does not allow (for example, adding a replica to an availability group on Express), SQL Server returns an error and the ClusterControl job fails with that error in the job log.
Limits on CPU and memory are applied silently by SQL Server. The instance keeps running but only uses the resources the edition allows.

Make sure your SQL Server edition and the number of licensed cores match your deployment. Microsoft's editions and supported features page lists the limits for each edition.

Standalone vs Always On availability group

ClusterControl supports two SQL Server topologies. If you specify one node, ClusterControl deploys a standalone instance (mssql_single). If you specify more than one, it deploys an Always On availability group (mssql_ao_async).

	Standalone	Always On availability group
Number of nodes	Exactly one node.	One primary and one or more secondary replicas. The first node you enter becomes the primary.
High availability	None. If the node goes down, the database is unavailable until the node is recovered.	Data is replicated from the primary to all secondary replicas. ClusterControl can fail over to a secondary replica if the primary fails.
Availability group	Not configured. `hadr.hadrenabled` stays disabled.	One availability group named `s9s_ag1`, created with `CLUSTER_TYPE = NONE`. A database mirroring endpoint named `s9s_hadr_endpoint`, listening on TCP port 5022 and authenticated with a certificate. Seeding mode is automatic, so new databases added to the availability group are copied to the secondaries by SQL Server.
Commit mode	Not applicable.	Asynchronous commit. The primary does not wait for secondaries to harden the transaction log before committing.
Readable secondaries	Not applicable.	Allowed (`ALLOW_CONNECTIONS = ALL`), except on Standard edition where they are set to `NO`.
Failover	Not applicable.	The SQL Server failover mode is `MANUAL`. ClusterControl handles failover itself: it can promote a secondary on request, or fail over automatically when cluster recovery is enabled.
Scaling	Add node is not supported, and a standalone instance cannot be converted into an availability group. Deploy a new Always On cluster instead.	Add or remove secondary replicas one at a time. The primary cannot be removed.
Backups	Taken on the single node.	Always taken on the current primary.

Since the availability group uses CLUSTER_TYPE = NONE, it is not tied to Pacemaker or Windows Server Failover Clustering, and SQL Server does not fail over on its own. Take note of the following assumptions:

No listener. ClusterControl does not create an availability group listener or a virtual IP. Your applications connect directly to the primary node, and they need to be pointed to the new primary after a failover. You can place a load balancer or DNS record in front of the nodes to handle this.
Hostnames. All nodes must be added using a hostname or FQDN, not an IP address. Inside the availability group, each replica is identified by its short hostname (the part before the first dot), and the endpoint URL uses that short name (tcp://<short_hostname>:5022). Every node must be able to resolve the other nodes by their short hostname. If a node is added with an FQDN, ClusterControl also renames the SQL Server instance (@@SERVERNAME) to the FQDN with sp_dropserver/sp_addserver and restarts SQL Server. If the rename fails, the job logs an error and the deployment continues.
Network. Port 1433 (client connections) must be reachable from ClusterControl and your applications. Port 5022 (availability group endpoint) must be open between all SQL Server nodes.
Same version. All nodes in an availability group must run the same major SQL Server version. Adding a node with a different version is rejected.
Automatic failover can lose data. Because replication is asynchronous, ClusterControl uses a forced failover (FORCE_FAILOVER_ALLOW_DATA_LOSS) when the primary is down. It picks the connected secondary with the lowest lag. Transactions that had not reached that secondary are lost.
Old primary is fenced. After a forced failover, if the old primary comes back, ClusterControl takes it offline, stops it, puts it into maintenance mode for 30 days, and raises a critical split-brain alarm. Review the node for data loss, then remove it from the cluster and add it back as a new replica.
Planned promotion. When you promote a secondary manually, ClusterControl temporarily switches the primary and the target replica to synchronous commit and waits until they are synchronized before it fails over, to avoid data loss. The original commit modes are restored afterwards.

Convert Always On availability group to standalone

ClusterControl has no built-in job to convert an Always On availability group cluster (mssql_ao_async) into a standalone SQL Server cluster (mssql_single). You can do it manually with the steps below. At the end, the current primary node becomes the standalone server and the secondary replicas are removed from the cluster.


Applies to	Microsoft SQL Server clusters with cluster type `mssql_ao_async`.
Downtime	Yes. Applications connected to a secondary replica, or through a load balancer or DNS record that points to the replicas, lose their connections. Plan a maintenance window.

Warning

Take a full backup and confirm it completed before you start.
Test the procedure on a non-production cluster first.

The steps use the following placeholders. Replace them with your own values:

Placeholder	Example	Description
`<cluster_id>`	`746`	The ClusterControl cluster ID.
`<primary>`	`mssql-db1.example.com`	The node to keep. It becomes the standalone server.
`<secondary>`	`mssql-db2.example.com`	A secondary replica to remove.
`<ag_name>`	`s9s_ag1`	The availability group name. ClusterControl always names it `s9s_ag1`.
`<database_name>`	`central`	A user database in the availability group.

The SQL statements in this procedure are run with sqlcmd on the SQL Server node, connected as the admin user you set during deployment (default SQLServerAdmin). The SA account is reserved for ClusterControl:

/opt/mssql-tools18/bin/sqlcmd -C -S localhost -U SQLServerAdmin

Step 1: Prepare the cluster

Check the cluster type and the node roles in ClusterControl. The cluster type must be mssql_ao_async. Note which node is the primary.
```
s9s cluster --list --long --cluster-id=<cluster_id>
s9s node --list --long --cluster-id=<cluster_id>
```

On the primary, confirm the replica roles and health:

SELECT ag.name AS ag_name,
       ar.replica_server_name,
       ars.role_desc,
       ars.connected_state_desc,
       ars.synchronization_health_desc
FROM sys.availability_groups ag
JOIN sys.availability_replicas ar ON ag.group_id = ar.group_id
JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;
GO

There must be exactly one PRIMARY, and it must be the node you want to keep. If the node you want to keep is a secondary, promote it first from Nodes → Actions (…) → Promote replica (see Promote replica), then run the query again.

On the primary, list the databases in the availability group and keep a note of them. They must all be ONLINE when the conversion is done.

SELECT DB_NAME(database_id) AS database_name,
       synchronization_state_desc,
       database_state_desc
FROM sys.dm_hadr_database_replica_states
WHERE is_local = 1;
GO

Take a full backup of the cluster. Go to ClusterControl GUI → Backups → Create backup, choose the cluster and set Backup method to Full. ClusterControl always takes SQL Server backups on the primary. See Create Backup.

Or use the CLI:
```
s9s backup --create \
    --cluster-id=<cluster_id> \
    --nodes=<primary>:1433 \
    --backup-method=mssqlfull \
    --wait
```

On the ClusterControl host, make a copy of the cluster configuration file and check the current values:

cp -p /etc/cmon.d/cmon_<cluster_id>.cnf /etc/cmon.d/cmon_<cluster_id>.cnf.bak.$(date +%F)
grep -E '^(cluster_type|type|mssql_server_addresses)=' /etc/cmon.d/cmon_<cluster_id>.cnf

The output looks similar to this:

cluster_type=mssql_ao_async
mssql_server_addresses=mssql-db1.example.com:1433,mssql-db2.example.com:1433

Point your applications to the primary node directly, and stop any traffic going to the secondaries. You can check the active sessions on the primary with:
```
EXEC sp_who2 'active';
GO
```

Step 2: Remove the secondary replicas

Remove each secondary replica from the cluster. ClusterControl removes nodes one at a time and does not allow you to remove the primary. For each node, it runs ALTER AVAILABILITY GROUP ... REMOVE REPLICA on the primary, stops SQL Server on the removed node, and removes the node from mssql_server_addresses in the cluster configuration.

ClusterControl GUIClusterControl CLI

Go to ClusterControl GUI → choose the cluster → Nodes.
Click Actions (…) → Remove node on a secondary replica.
Choose how to handle the service on the node, then click Remove. See Remove Database Node.
Repeat for every secondary replica.

Run this once for every secondary replica:

s9s cluster --remove-node \
    --cluster-id=<cluster_id> \
    --nodes="<secondary>:1433" \
    --wait --log

Check that only the primary is left, both in ClusterControl and in SQL Server:

s9s node --list --long --cluster-id=<cluster_id>

SELECT replica_server_name FROM sys.availability_replicas;
GO

Note

The copies of the databases on the removed nodes stay in RESTORING state. ClusterControl no longer manages those nodes. Drop the databases there, or decommission the servers, when you no longer need them.

Step 3: Drop the availability group

On the primary, drop the availability group.

USE master;
GO
DROP AVAILABILITY GROUP [s9s_ag1];
GO

Optionally, drop the database mirroring endpoint as well, since a standalone server does not use it:

DROP ENDPOINT [s9s_hadr_endpoint];
GO

Warning

Do not drop the S9sDbmEncryptCert certificate or the database master key. ClusterControl still uses the certificate to encrypt backups on the standalone server, and you need it to restore backups taken earlier.

Check that the availability group is gone. Both of these queries must return no rows:

SELECT name FROM sys.availability_groups;
SELECT DB_NAME(database_id) AS database_name FROM sys.dm_hadr_database_replica_states;
GO

Then check that all user databases are ONLINE and MULTI_USER:

SELECT name, state_desc, user_access_desc FROM sys.databases WHERE database_id > 4;
GO

Step 4: Change the cluster type in ClusterControl

Change the cluster type to mssql_single:

s9s cluster --change-config \
    --cluster-id=<cluster_id> \
    --opt-name=cluster_type \
    --opt-value=mssql_single

Then check the configuration file on the ClusterControl host:

grep -E '^(cluster_type|type|mssql_server_addresses)=' /etc/cmon.d/cmon_<cluster_id>.cnf

The output should look like this:

cluster_type=mssql_single
mssql_server_addresses=mssql-db1.example.com:1433

Note

type is an alias of cluster_type. If the file has a type= line that still says mssql_ao_async, edit it by hand so both say mssql_single. mssql_server_addresses must list only the primary node.

Step 5: Restart the ClusterControl controller

ClusterControl reads the cluster type from the configuration file when the controller starts, and updates the cluster record in the cmon database at the same time. Restart the controller to load the cluster as a standalone server:

systemctl restart cmon
systemctl status cmon --no-pager

Check the cluster:

s9s cluster --list --long --cluster-id=<cluster_id>
tail -n 100 /var/log/cmon_<cluster_id>.log

Within a minute or two, the cluster type should show mssql_single and the state STARTED, with no errors in the log.

Step 6: Validate the standalone cluster

Check	Command	Expected result
No availability group metadata	`SELECT name FROM sys.availability_groups;`	No rows.
No HADR database states	`SELECT DB_NAME(database_id) FROM sys.dm_hadr_database_replica_states;`	No rows.
Databases are online	`SELECT name, state_desc, user_access_desc FROM sys.databases WHERE database_id > 4;`	All `ONLINE` and `MULTI_USER`.
Applications can connect	`EXEC sp_who2 'active';`	Application sessions on the user databases, with nothing blocked or stuck in `SUSPENDED`.
Cluster state	`s9s cluster --list --long --cluster-id=<cluster_id>`	`mssql_single`, `STARTED`.
Node list	`s9s node --list --long --cluster-id=<cluster_id>`	One node, online.
Backups work	Create a new full backup (`mssqlfull`).	The backup job completes.

Restoring a backup after the conversion

If you restore a full backup on the standalone server, ClusterControl drops the existing databases included in the backup before it restores them. If a database cannot be dropped, for example because sessions are still connected to it, the restore job fails with Could not drop database <database_name>. Cannot restore backup. In that case, drop the database manually and run the restore again:

USE master;
GO
ALTER DATABASE [central] SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
GO
DROP DATABASE [central];
GO

Warning

Only drop a database after you have confirmed that the backup you are going to restore is valid.

If a restored database is left in SINGLE_USER mode, set it back with ALTER DATABASE [central] SET MULTI_USER;.

Default configuration applied by ClusterControl

During deployment, ClusterControl performs the following configuration on each SQL Server node.

Operating system and packages

Adds the Microsoft package repository and installs mssql-server and mssql-tools18 (plus libicu on RHEL-based systems). The EULA is accepted on your behalf.
Optionally sets SELinux to permissive and disables the firewall, depending on the deployment options.
Installs the Microsoft ODBC driver (msodbcsql18) on the ClusterControl host so it can connect to SQL Server.
Refuses to install if the data directory already exists and is not empty, to avoid overwriting an existing installation.

SQL Server settings (set with mssql-conf, stored in /var/opt/mssql/mssql.conf)

Setting	Value
`filelocation.defaultdatadir`	`/data/microsoft`, or the data directory you specify.
`filelocation.defaultbackupdir`	`/data/backups`, or `<datadir>/backups` if you specify a custom data directory.
`filelocation.errorlogfile`	`/var/log/mssql/errorlog`, or `<datadir>/log/errorlog` if you specify a custom data directory.
`memory.memorylimitmb`	About 80% of the node's memory. ClusterControl rounds the total memory to the nearest power of two before calculating it.
`sqlagent.enabled`	`1`. SQL Server Agent is enabled, and so are Agent XPs.
`hadr.hadrenabled`	`1`, for Always On availability groups only.

Users

User	Purpose
`SA`	Used by ClusterControl to manage the instance. ClusterControl generates the SA password and stores it in the cluster configuration file on the ClusterControl host (`/etc/cmon.d/cmon_<cluster_id>.cnf`).
Admin user (default `SQLServerAdmin`)	The admin user you enter in the deployment wizard, created as a member of the `sysadmin` role. If you leave the password empty, ClusterControl generates one and prints it in the job log. Save the password and delete the job afterwards.
`mssql_exporter`	Monitoring user with `VIEW ANY DATABASE`, `VIEW SERVER STATE` and `VIEW ANY DEFINITION` permissions. Used by the SQL Server exporter that ClusterControl installs for monitoring.

The sa and mssql_exporter accounts cannot be dropped from ClusterControl. Avoid the characters { } ( ) [ ] @ ; = in the admin user name and password, because they break the ODBC connection string that ClusterControl uses. If you use them, ClusterControl replaces each one with % and only notes this in the job log, so the login is created with a different name or password than the one you entered.

ClusterControl also creates a database named admindb and installs a few helper stored procedures (such as sp_WhoIsActive) that it uses for monitoring and query statistics. For an availability group, admindb is backed up once and added to s9s_ag1.

Database master key and certificate

On a new deployment, ClusterControl creates a database master key and a certificate named S9sDbmEncryptCert in the master database. The same certificate is used for two things:

Encrypting every backup taken by ClusterControl (ENCRYPTION (ALGORITHM = AES_256, SERVER CERTIFICATE = S9sDbmEncryptCert)).
Authenticating the availability group endpoint s9s_hadr_endpoint between replicas.

For an availability group, the certificate is created on the primary, exported to <datadir>/s9s_dbm_certificate.cer and .pvk, copied to every replica and imported there. The master key password and the private key password are generated by ClusterControl and stored in the cluster configuration file as mssql_master_key_password and mssql_backup_cert_password. Right after the deployment, ClusterControl also runs a one-time backup of the certificate with a retention of 10 years.

The certificate is valid for one year

ClusterControl creates the certificate without an expiry date, so SQL Server uses its default validity of one year from the creation date. ClusterControl does not monitor the certificate and does not raise an alarm before it expires. Add a reminder to renew the certificate before it expires.

To check when the certificate expires, run the following on each node:

/opt/mssql-tools18/bin/sqlcmd -C -S localhost -U SQLServerAdmin \
    -Q "SELECT name, start_date, expiry_date FROM sys.certificates WHERE name LIKE 'S9sDbmEncryptCert%'"

Renew the backup certificate

Renewing creates a new certificate and switches ClusterControl over to it. It works for both standalone and Always On clusters. Use the ClusterControl CLI:

s9s cluster --renew-cert \
    --cluster-id=<cluster_id> \
    --log

The following actions are performed:

A new certificate named S9sDbmEncryptCert_<timestamp> is created on the primary node.
The certificate and its private key are exported to the backup directory as <name>.cer and <name>.pvk.
For an availability group, the files are copied to every secondary replica and the certificate is imported there. The endpoint s9s_hadr_endpoint is then stopped, switched to the new certificate and started again, first on the secondaries and then on the primary.
The cluster configuration is updated, so all new backups are encrypted with the new certificate.

Note

For an availability group, replication pauses briefly while the endpoint restarts with the new certificate. Run the renewal during a quiet period.
The old certificate is not dropped. SQL Server still needs it to restore backups that were encrypted before the renewal, so do not remove it while you still have those backups.
Creating the new certificate needs a database master key in master. ClusterControl creates one on new deployments, but not on imported clusters, so the renewal fails on an imported standalone server that has no master key.
To restore an encrypted backup on a server outside the cluster, you need the matching certificate and private key files, and the private key password (mssql_backup_cert_password) from the cluster configuration file. Keep copies of these in a safe place.

Verify the renewal

The renewal job does not roll back if it fails partway, so check the result after the job finishes. Run the SQL queries with sqlcmd on the SQL Server node, connected as your admin user (default SQLServerAdmin). Checks 4 and 5 apply to availability groups only.

Check that the job finished. The status must be FINISHED, and the log should list the new certificate name with no FAILURE lines:
```
s9s job --list --cluster-id=<cluster_id> | tail -5
s9s job --log --job-id=<job_id>
```
On the ClusterControl host, check that the cluster configuration points to the new certificate. It should show mssql_cert=S9sDbmEncryptCert_<timestamp>:
```
grep '^mssql_cert=' /etc/cmon.d/cmon_<cluster_id>.cnf
```
On every node, check that the new certificate exists. You should see the previous certificate and the new S9sDbmEncryptCert_<timestamp>, with an expiry_date one year from the renewal date:
```
SELECT name, start_date, expiry_date
FROM sys.certificates
WHERE name LIKE 'S9sDbmEncryptCert%'
ORDER BY start_date;
GO
```

On every node, check that the endpoint uses the new certificate and is running. Each node should show s9s_hadr_endpoint, STARTED, and the new certificate name:

SELECT e.name AS endpoint, e.state_desc, c.name AS certificate
FROM sys.database_mirroring_endpoints e
JOIN sys.certificates c ON e.certificate_id = c.certificate_id;
GO

On the primary, check that all replicas reconnected after the endpoint restart. Every replica should be CONNECTED and HEALTHY:

SELECT ar.replica_server_name, ars.role_desc,
       ars.connected_state_desc, ars.synchronization_health_desc
FROM sys.availability_replicas ar
JOIN sys.dm_hadr_availability_replica_states ars ON ar.replica_id = ars.replica_id;
GO

On every node, check that the exported certificate files exist in the backup directory (/data/backups by default) and are owned by mssql. The job does not stop if the export fails, so do not skip this check:
```
sudo ls -l /data/backups/S9sDbmEncryptCert_*.cer /data/backups/S9sDbmEncryptCert_*.pvk
```

Take a new full backup, then check on the primary that it was encrypted with the new certificate. The latest backup should show S9sDbmEncryptCert_<timestamp>, while older backups show the previous certificate:

SELECT TOP 5 b.database_name, b.backup_finish_date, c.name AS encrypted_with
FROM msdb.dbo.backupset b
LEFT JOIN master.sys.certificates c ON b.encryptor_thumbprint = c.thumbprint
ORDER BY b.backup_finish_date DESC;
GO

If a check fails, review the job log, fix the cause, and run the renewal again. Each run creates a new certificate with a new timestamp.

Known limitations

Upgrades, rolling restarts, and changing the SSL/TLS certificate are not supported for SQL Server clusters. SQL Server uses its own self-signed certificate for client encryption.
Only asynchronous commit mode is deployed. New replicas added later are also asynchronous.
ClusterControl manages a single availability group, s9s_ag1. Other availability groups on the same nodes are not managed.
No availability group listener is created, and SQL Server does not fail over automatically on its own. Failover relies on ClusterControl cluster recovery, which needs a valid ClusterControl license.
Automatic failover is a forced failover and can lose transactions that were not yet replicated.
Add node is not supported on a standalone instance, so it cannot be converted into an availability group.
Nodes can only be added or removed one at a time, and the primary cannot be removed.
The endpoint port 5022 is fixed and cannot be changed.
A SQL Server product key cannot be entered during deployment. See Supported SQL Server licenses.
ClusterControl does not check SQL Server edition limits before running a job. See What happens when the license limit is exceeded.
The backup certificate expires after one year and is not monitored. See Renew the backup certificate.
For imported clusters, ClusterControl does not create the master key and certificate, so backups are not encrypted unless a certificate with the name stored in the cluster configuration (mssql_cert) already exists on the node.
Backups cannot be stored on the ClusterControl host. They are stored on the node that takes the backup.
Restoring to an availability group must be done on the primary. Databases that belong to more than one availability group cannot be restored. System databases (master, msdb, model) are never restored.
A database that is part of the availability group cannot be dropped until it is removed from the group.
Importing SQL Server clusters with the ClusterControl CLI is not supported. Use the GUI instead.