Skip to content

ClusterControl Agent

Also known as the query monitoring agent. Deploys agents for monitoring the database nodes, particularly the database queries. It is only available for MySQL and PostgreSQL database systems.

Features you can use with Query Monitor/Agent are the following:

  • Install/remove query monitor agents on each database node.  You can remove these agents later on.
  • Start/stop collecting query stats with the agents.
  • New ‘Query Workload’ overview showing query digests, latency, throughput, and concurrency with a scatter chart.

Query Monitor Agents

Log in to your ClusterControl GUI → choose a database cluster → Performance → Query Monitor/Agents.When you click the Install monitoring agent button, ClusterControl will set up and install an agent called cmnd to all of the database nodes in that particular cluster.

For CLI, we will use the s9s cluster option: Deploys CMON agents to all nodes in cluster ID 1. (This enables TopQuery Monitoring functionality for the cluster nodes.):

s9s cluster \
        --deploy-cmonagents \
        --cluster-id=1 \
        --print-request \
        --log \
        --wait

Uninstalls CMON agents from specified nodes in cluster ID 1. (If no node is specified, CMON agents will be uninstalled from all nodes in the cluster.)

s9s cluster \
        --uninstall-cmonagents \
        --cluster-id=1 \
        --nodes=10.67.199.164 \
        --print-request \
        --log \
        --wait

Agent Configuration

The agent has the following configuration which you can find in /etc/cmnd.conf. This is for the agent configuration.  /etc/cmnd.d directory for the Top Query configuration. The agent configuration file /etc/cmnd.conf has all the details you can find here. By default, these are the following options that are set:

$cat /etc/cmnd.conf | sed -e 's/#.*$//' -e '/^$/d' -e '/^$/N;/^\n$/D' | sed '/^\t*$/d'

[cmnd]
data_directory = "/var/lib/cmnd"
log_file = "/var/log/cmnd.log"
pid_file = "/var/run/cmnd.pid"
plugin_names = [
"libpluginMySql.so",
"libpluginSqlite.so",
"libpluginPgSql.so"
]
port = 4433
cert_file = "/etc/ssl/cmnd/cert.pem"
key_file = "/etc/ssl/cmnd/key.pem"

You can change the port to your desired port number based on the firewall policy you have in your organization. So take note, you need to have port 4433 open by default to make it work properly.

You can find more of its configuration files under these files/directories:

$ find /etc/ -name "cmn*"

/etc/ssl/cmnd
/etc/logrotate.d/cmnd
/etc/cmnd.conf
/etc/cmnd.d
/etc/cmn.passwd

The Query Monitor Agents are installed using systemd, allowing you to start and stop the service using the systemctl command. To determine if the agents are running, you can do the following:

systemctl status cmon-daemon

Or verify it with the netstat just like below:

$ netstat -ntlvp46 | grep cmnd

tcp 0 0 0.0.0.0:4433 0.0.0.0:* LISTEN 8456/cmnd

Query Monitoring Workflow

The agent collects samples from the performance_schema (MySQL/MariaDB) or pg_stat_statements (PostgreSQL/TimescaleDB). These metrics which is collected by a database client and send the query inquiry to the agent. Then, the agent that reads the input data will produce the query statistics and make it available to the Query Monitor’s Overview dashboard query metrics. That means all collected information from the Query Monitoring shall depend upon the following conditions:

  1. The reply should be sent in a timely fashion no matter how big the input data is. Should the computer be very slow or the processing is too complicated, the query statistics should be produced and made available in a progressive manner so a responsive UI can be implemented.

  2. The agent should be able to update the query statistics on itself when the input data is updated by the database server. A simple query or RPC call is invoked so that the agent keeps monitoring the input data and keeps the data updated.

  3. The query statistics should be according to the query inquiry and the input data. In simpler words, the produced statistical data should be in sync with the input data and the query the user sent. Some query statistics may be derived from the input data.