- In Gotion’s production instance, TDengine ingests millions of records per second and compresses data to 10% of its original size.
- Gotion’s team can focus on their business requirements with the fully managed TDengine Cloud taking care of database administration and management tasks.
- Private links to TDengine Cloud in AWS significantly reduce network latency and bandwidth costs.
In one of our energy storage projects, we needed a way to monitor the safety of our batteries in real time. The data we were collecting — battery charging and usage data, current, voltage, and so on — were all classic examples of time-series data. With massive amounts of time-series data expected to come in from our users, we decided to deploy a purpose-built time-series database.
Background
We deployed TDengine 2.x on premises in 2022, but coordinating with our maintenance teams proved to be difficult. At the time, we were using the open-source TDengine OSS, which is not managed and does not include dedicated support, and our staff were spending much time learning and deploying the product. It was then that we learned of TDengine Cloud, and what excited us most about the platform was the fact that it is fully managed.
With TDengine Cloud, we received the same feature set as TDengine Enterprise but were freed from the burdens of database deployment, optimization, backup, and disaster recovery — allowing us to focus all our attention on our core business. As we were still in the early stages of the project, we found that the Starter package listed on TDengine Cloud was a good fit for our requirements.
We created a supertable in TDengine for each type of battery in the project, and then created a table for each battery within those supertables. The supertables each have over 100 columns, including metrics such as electricity usage and charging status. Our batteries generate a data point every 10 seconds 24 hours a day, which is written to TDengine. We can easily view the metadata for our tables and supertables in the Data Explorer within TDengine Cloud.
Data Ingestion Performance
TDengine is designed based on the characteristics of time-series data and innovates in several areas — columnar compression, flexible tags, and one table per device, for example. By using TDengine, we could avoid issues such as write locks, low compression ratios, and duplicated static data.
The data generated by our storage devices is transmitted over MQTT into the system, where business data is forwarded to PostgreSQL and time-series data — device-generated data as well as logs and statuses — is sent to TDengine. Each system in the middle tier uses the data stored in these databases for analysis and computation but may also send commands directly to devices. Finally, the data is displayed on management platforms.
During our testing, TDengine was able to ingest millions of data points per second and compress our datasets to 10% of their original size. These results were reflected in production as well.
Query Performance
We perform many kinds of queries on our data in order to provide the best user experience and after-sales service. These can include monitoring battery health, analyzing trends in battery usage, and estimating product lifetime, among others.
Several typical queries are described as follows:
- We use session windows to calculate the sum of the power output of a single device in intervals less than or equal to 1 minute:
SELECT FIRST(ts) firstTs, LAST(ts) lastTs, COUNT(*) countVal, SUM(input_total_power) totalPower FROM device_data_58CF7920B63C WHERE ts >= '2023-08-09T00:00:00.000Z' AND ts< '2023-08-10T00:00:00.000Z' SESSION(ts, 1m);
- We use time windows (
INTERVAL) to query trends in device power output with theOFFSETparameter to take time zones into account:SELECT _WSTART AS ts, AVG(inverter_power) avgVal, MIN(inverter_power) minVal, MAX(inverter_power) maxVal FROM gendome.device_data_58CF7920B63C WHERE inverter_power > 0 AND ts >= '2023-08-09T00:00:00.000Z' AND ts< '2023-08-10T00:00:00.000Z' INTERVAL (1d, 960m) FILL (NULL);
Because our applications are in the same AWS region as our TDengine Cloud deployment, we can create private links between them and TDengine so that our data is not transmitted over the Internet. This greatly decreases ingestion latency and also reduces costs associated with network traffic.
Conclusion
TDengine Cloud helps us run our project at minimal cost thanks to its reasonably priced, fully managed enterprise service. It has greatly improved our efficiency and given us the flexibility to scale out at any time that our requirements might demand it. Going forward, we plan to explore the capabilities of TDengine’s data sharing and stream processing features as well.
About Gotion
Gotion is a leading global solutions provider in the renewable energy sector. The company focuses on researching, producing, and marketing lithium-ion batteries and electricity storage.
