With industries ranging from IoT to manufacturing generating and collecting a constantly increasing amount of time-series data, the growth of the time-series database (TSDB) sector over the past five years has not come as a surprise. This popularity has resulted in a large number of solutions coming on the market, sometimes making it difficult to choose the best time-series database for a certain business scenario.
About InfluxDB
InfluxDB was originally released in 2013, making it one of the earliest of the new generation of open-source time-series databases. It is the most popular TSDB at present and has been widely deployed in IT monitoring and DevOps scenarios.
About TDengine
TDengine is a next generation data historian purpose-built for Industry 4.0 and Industrial IoT. At its core is the open-source time-series database TDengine OSS, which differentiates itself with high performance, a distributed cloud native architecture, and a comprehensive design including built-in caching, data subscription, and stream processing.
Which Is Best for You?
Both InfluxDB and TDengine are among the top 10 time-series databases on the market, and in terms of time-series data storage and processing, both are vastly superior to general-purpose solutions such as relational databases, NoSQL databases, and big data platforms like Hadoop. Some of the main differences between the two systems are described as follows:
- Performance is the main reason that customers deploy purpose-built time-series databases, and according to benchmark testing based on the open-source TSBS framework, TDengine OSS significantly outperforms InfluxDB in all key areas — data ingestion rate, query response time, and disk space usage — while using fewer server-side resources.
For the full report, see IoT Performance Comparison. Note that InfluxData claims to have significantly improved performance with InfluxDB 3.0, but the open-source edition has not yet been released.
- System complexity: In time-series data processing scenarios, business requirements often include the deployment of caching, message queue, and stream processing products along with the time-series database. For example, an InfluxDB-based solution may deploy Kafka to collect incoming data and then send that data to Redis for caching and Spark for stream processing in addition to InfluxDB for storage.
TDengine includes these three important functions as built-in components, greatly simplifying your overall architecture as shown in the preceding figure. This makes development easier and faster, because applications do not need to receive data from different systems, and reduces costs associated with personnel training and fault location for these complex systems. In addition, because TDengine’s caching, data subscription, and stream processing components are used only for time-series data processing, they can be optimized beyond the limits of general-purpose components.
- Data sources: InfluxDB ingests data through Telegraf, one of the most popular data collection agents, and there are a vast number of Telegraf plugins that enable the ingestion of data from sources commonly used in IT monitoring and DevOps environments. TDengine, on the other hand, is intended for industrial data sources, and includes connectors for modern industrial protocols like MQTT and OPC UA as well as traditional historians such as PI System and Wonderware Historian. To guarantee data quality and handle ingestion from different sites or platforms, TDengine includes powerful extract, transform, and load (ETL) capabilities, supporting mapping, transformation, and filtering out of the box. Note that TDengine can also ingest data from Telegraf if desired.
- Clustering: TDengine OSS includes clustering so that the open-source community can benefit from high availability. Unfortunately, this feature is no longer included in the open-source version of InfluxDB.
- Data contextualization: Both InfluxDB and TDengine support associating time series with tags to provide context. In InfluxDB, however, adding, deleting, or updating a tag will be considered a new time series, meaning that the context is not maintained. TDengine also supports hierarchical tags and can even import the hierarchy from PI System’s Asset Framework (AF) automatically.
- Enterprise features: TDengine has built-in site-to-site data replication, incremental data backup, IP whitelists, views with access control, tiered storage including S3 support, and data encryption both in transit and at rest. These features make TDengine a more suitable choice for enterprise customers.
Conclusion
InfluxDB and TDengine both have rich feature sets and are great choices for processing time-series data. The best choice may depend on your use case — InfluxDB is more commonly used in IT monitoring scenarios, while TDengine is designed for industrial data and has better support for hierarchies and legacy industrial systems. TDengine also has significant advantages in terms of performance, giving it the edge in speed- or cost-sensitive environments.
In fact, InfluxDB has undergone significant changes over its lifetime, reimplementing its entire storage engine and query language with version 3.0 to address performance issues, especially with high-cardinality datasets. While these may be steps in the right direction, they can result in painful migration processes for existing customers who are forced to rebuild applications and dashboard each time InfluxData pivots. TDengine, on the other hand, has always supported standard SQL and was designed from the beginning to support high scalability and massive datasets.
InfluxDB has a large community and good documentation resources, but TDengine’s user base is growing rapidly as well. If you’d like to see whether TDengine is a good choice for you, download TDengine OSS for Linux or macOS today and see how quickly you can set up an instance and run your queries. TDengine OSS delivers all core components needed for time-series data processing, including clustering, and you can quickly create large-scale sample datasets with our taosBenchmark tool to simulate your actual use cases. And if you prefer a fully managed experience, TDengine Cloud offers a free month of service for new registrations, with no credit card or payment information required.
