As the number of sensors and devices in the Internet of Things (IoT) increases by a CAGR of ~25%, i.e., doubles every 3 years, a critical question that arises is how to ingest, store and analyze the massive amounts of data generated and transmitted. Given the time-series nature of this data, the natural solution is a time-series database. There are several alternatives including monitoring solutions such as InfluxDB, Prometheus and TimeScale with proprietary query languages or which are general purpose but support time-series data.
However, what is needed is a time-series database (TSDB) built specifically for, and optimized for, the taxing requirements of IoT. TDengine is exactly this – a high-performance, scalable time-series database, with a standards-based query language, purpose built to meet the rigorous demands of IoT.
In recent years, NoSQL databases have been touted as a good solution for time-series, and other kinds of data. While there is no doubt that NoSQL databases have a role to play in IoT infrastructure, SQL databases, a robust technology based on the sound mathematical theory of relational calculus, are to this day, used in industries with extremely demanding requirements. SQL databases provide stable, well-understood technology and of course, a standard query and data definition language that is fairly uniform across platforms. SQL is both an ANSI and ISO standard. Several tools exist to create SQL queries even for those not well-versed in SQL. TDengine chooses to provide SQL as the query language and also provides the TDengine shell for quick access to the database for ad hoc SQL queries. Other time-series platforms have proprietary query languages which introduces an unnecessary learning curve and also makes interoperability and query reuse impossible.
Unlike other time-series database engines, TDengine was written from the ground up and is heavily optimized, specifically for IoT environments. One of the basic premises of TDengine is that time-series data has certain characteristics and these characteristics have been taken into account for optimization. TDengine’s designers have added some innovations to ensure that TDengine delivers very high performance out of the box. Some of these include the concept of the “vnode”, allowing both row and columnar format storage and using optimized indices for storage in RAM and on hard-disk. TDengine also introduces the innovative idea of a supertable. A supertable allows easier management and definition of data for the same type of devices and also allows flexibility when devices change or requirements change. Another unique feature of TDengine is using one table per device. This allows performance optimizations which make TDengine faster than other database engines. Additionally, database architects are given sufficient flexibility to design the schema in ways that improve performance for their specific application(s). This combination of off-the-shelf performance as well as the flexibility of application specific enhancements, ensures that TDengine is one the most high-performance time-series databases for IoT.
When it comes to scalability, TDengine delivers a highly scalable platform. This is accomplished firstly because TDengine was designed to be a distributed platform for IoT. Through node virtualization and automatic load balancing TDengine makes very efficient use of compute resources which increases scalability. Scalability is further enhanced with carefully executed data partitioning and data sharding. Clustering is built-in and it is very simple to configure and deploy additional nodes to a cluster. It is important to note that unlike other open-source time-series database platforms, TDengine provides scalability features even in the open-source edition.
TDengine is a very versatile platform designed to be used both by those embarking on their IoT journey as well as those in the middle of their IoT journey. It is an easy and scalable platform for those starting out on their IoT journey with simple APIs, integrations and easy-to-use tools. For those in the middle of their IoT journey who are looking for an alternative to existing time-series databases, TDengine is a very powerful production-ready, and even carrier-grade IoT platform with advanced configurations, and easy to integrate into existing infrastructure.
