Solar Panel Monitoring: MQTT Ingestion

MQTT is a data protocol commonly used for communication in modern IIoT environments, including many renewable energy sites. Devices can send the data they collect to an MQTT broker, potentially via an IoT gateway, so that it can be used in higher-level applications and algorithms. TDengine, being a time-series database designed for the IIoT, includes a built-in MQTT connector that can be configured without writing a single line of code, making it easier than ever to transfer your operations data into persistent storage and share it with your business and analytics apps and teams.

This article describes how use TDengine’s MQTT connector to ingest and monitor data generated by solar panels at different sites. In this example, the data is sent from an MQTT broker to TDengine Cloud and then visualized in Grafana.

Video Tutorial

Introduction

For this demonstration, a simulator writes data to a public MQTT broker hosted by EMQX. The simulated data is for three solar farms. Each farm has 10 strings, and each string has 10 panels. Data records are transmitted in JSON format and look as follows:

{
    "ambienttemperature_c": "26.29",
    "windspeed_mps": 8.02,
    "poweroutput_kw": "350.84",
    "current": "7.17",
    "voltage": "48.92",
    "tstamp": "2024-07-01T21:54:48-07:00",
    "panelid": "panel4",
    "string": "string1",
    "site": "solarfarma"
}

As you can see, these records contain various metrics collected by each solar panel as well as static attributes for the site, string, and panel ID. Note that this simulator is simulating data for coordinates in California and after sunset there is no data except for windspeed and ambient temperature.

Create an MQTT Data Source

To follow along, it is easiest to use a Starter account on TDengine Cloud. If you do not already have an account, you can create one easily. No credit card is required and the instance is active for a month.

Once you have logged into your account, click Data In in the main menu on the left.

On the Data Sources tab, click Add Data Source.

Configure the Data Source and Database

1. In the Name field, enter a name for your data source.
2. From the Type drop-down list, select MQTT.
3. Determine whether to specify a connection agent.
   • If your MQTT broker is public, leave the Agent field blank.
   • If your MQTT broker is behind a firewall, select an agent on the same network as your MQTT broker. If you have not created an agent in advance, click Create New Agent and follow the instructions displayed on screen. For more information, see Install Connection Agent.
     
     In this example, the MQTT broker is accessible on public networks, so an agent is not used.
4. Specify a database in TDengine to store the data ingested from your MQTT data source.
   Note that you must specify a database that is empty and whose CACHEMODEL parameter has been set to both.
   
   If a database has not been created in advance, click the Create Database button.

For more information about creating databases in TDengine, see Create a Database.

Configure the Connection

1. Under Connection Configuration, enter the hostname and port of your MQTT broker.

2. Under Authentication, enter the username and password to connect to your MQTT broker and topic.
   In this demonstration, the MQTT broker and topic used do not require authentication and the fields are left blank.

Configure Data Collection

Enter the information shown as follows. Note that each client ID must be unique. You must enter a different client ID when you are trying this out.

After you have entered the required information, click Check Connection. The message “Your data source is reachable.” indicates that your data source has been configured correctly.

Note that the “topic” is a string which the MQTT broker will use to filter the messages sent to a particular client. QoS is Quality of Service defines the level of delivery guarantee. In this case, we use QoS level 0, which is “at most once” or “fire and forget”, but in a production environment you may use 1 or 2 depending on network reliability and the requirements of your application.

Transform Your Data

To configure the “Payload Transformation” section, you can copy and paste the JSON message from the top of this blog.

Watch the video snippet below that shows you how to configure the payload and the supertable.

Once the data source is created, you will notice that that status says “Queued”. You can just hit the refresh icon above the Status heading and see that it will say “Running”.

Note that in this case, this schema will create a subtable for each panel. The supertable is set up to represent solar panels and the contextualization of each panel is present in the tags which tell us which site and in which string the panel is deployed.

View Data in TDengine

Click Explorer in the main menu to the left to see the database and verify whether it is ingesting data from the MQTT data source.

If you click on a subtable, you can see it has data in it.

Now that we are done with setting up an MQTT data source, in the next article we will setup Grafana dashboards to visualize this data in real time.