Lightweight, Open-Source IT Infrastructure Monitoring

Whether you’re a sysadmin in a datacenter or just running a server at home, monitoring your infrastructure is an important task that ensures you can respond quickly to any incidents that may occur.  This walkthrough will show you how to set up a basic monitoring system in only a few minutes using only open-source software.

Introduction

There are three basic components to a monitoring system — data collection, persistence, and visualization — and fortunately, there are great open-source choices for all of these. For data collection, we’ll use Telegraf in this walkthrough, though you have a number of good alternatives such as collectd. For visualization, we’ll go with Grafana, which has more or less become the gold standard in observability. And considering that the metrics we’re monitoring are time-series data, a time-series database is naturally the best fit for persisting that data.

In this article, we’ll introduce TDengine OSS — a free, open-source time-series database released under the AGPLv3. It includes all the components that you need for time-series data processing, including clustering, and is developed in the open on GitHub. Most importantly, you can install TDengine and integrate it with Telegraf and Grafana in only a few minutes.

Install TDengine

To get started, download the TDengine OSS installation package for your Linux distribution and install it on your local machine. For detailed instructions, see the TDengine documentation. Once the installation has completed, make sure that the taosd and taosadapter services are running:

sudo systemctl status taosd
sudo systemctl status taosadapter

Run the taos command to open the TDengine CLI and create a user to own the data collected by Telegraf — in this example, named monitoringuser — with a secure password.

CREATE USER monitoringuser PASS '<your-password>';

Now that TDengine is up and running, you can set up your collection and visualization tools.

Install Telegraf and Grafana

To install Telegraf, refer to the official documentation. Once the installation has completed, modify the Telegraf configuration file and add an output plugin to send data to TDengine.

[[outputs.http]]
url = "http://localhost:6041/influxdb/v1/write?db=monitoringdb"
method = "POST"
timeout = "5s"
username = "monitoringuser"
password = "<your-password>"
data_format = "influx"

While you’re modifying the configuration file, make sure to comment out any output plugins that you aren’t using, or Telegraf might give you an error. The configuration described here uses a TDengine server deployed locally and creates a database called monitoringdb to store the data collected by Telegraf.

After you update the configuration, start Telegraf and verify that it’s sending data to TDengine — just open the TDengine CLI and run the following commands

USE monitoringdb;
SHOW TABLES;

Finally, install Grafana according to the official documentation. Once the installation has completed, open Grafana and log in. Then install the official plugin for TDengine and add your TDengine deployment as a data source. Now that data from TDengine can be used in your Grafana deployment, add the Telegraf System Dashboard to visualize it.

Conclusion

Monitoring systems contain data acquisition, persistence, analytics, and visualization components. When monitoring large data sets, the bottleneck in terms of system performance is usually in the persistence layer. By deploying a purpose-built time-series database like TDengine as your data storage platform and then integrating with data acquisition and visualization components like Telegraf and Grafana, you can quickly create an IT infrastructure monitoring system that provides the information you need.