The design of TDengine is based on the assumption that a single hardware and software system or unit is unreliable, and that any single computing node cannot provide sufficient computing power and storage capacity to process massive data. Therefore, from the first day of its development, TDengine has been designed with a horizontal scaling and high availability architecture. By partitioning data, and using virtual node (vnode) technology, the processing capacity of the system is guaranteed to be horizontally scalable. If you want to increase the processing power of the system, you only need to add new nodes.
Furthermore, in August 2020, the TDengine team open-sourced the cluster feature.
TDengine partitions big data in two dimensions to achieve scalability – data collection points and time.
Sharding: In the design and implementation of TDengine, a cluster has multiple nodes, each node can have one or more virtual nodes (vnode), each virtual node stores the data of a certain number of data collection points, and the data of a data collection point is always stored in only one vnode. This way, if there are many data collection points, the data from these data collection points will be distributed to multiple vnodes and distributed in multiple nodes. When data is ingested, the TDengine client directly writes the data to the corresponding vnode, thereby achieving horizontal scaling of data ingestion. For the query of the data of a single data collection point, horizontal scaling is obvious. The more nodes there are, the greater the throughput rate. For aggregation queries, the query request will be sent to the corresponding vnodes first, the vnodes will complete the aggregation operation, then the client will aggregate the query results from multiple vnodes for a second time. Because the number of vnodes is limited, aggregation queries require little computation on the client side, so the horizontal scalability capability of aggregation queries is achieved.
Partitioning: In addition to sharding the data, TDengine also divides the time series data stored in a vnode according to time periods. The data of each time period must be saved together, and the data of different time periods will not overlap. The time period can be one day or multiple days, which is defined by the user. Dividing time series data by time period has many advantages. When querying data, the file to be searched can be directly located according to the time period, thereby speeding up the query. On the other hand, data retention policies can be efficiently implemented. If the data that has been retained for the longest period of time is exceeded, the files corresponding to a period of time can be deleted directly. In addition, when data is segmented according to time periods, it becomes much easier to achieve multi-level storage and reduce storage costs further.
TDengine also provides high availability of the system through virtual node group technology. Vnodes on different nodes can form a virtual node group. The data in this virtual node group is synchronized through the Master-Slave structure to ensure the consistency of data in this virtual node group. Data writes can only be performed on the master, but queries can be performed on both the master and slave simultaneously. If the master node fails, the system automatically selects a new master node and continues to provide services and ensures high availability.
For more detailed explanation of the clustering design, please check TDengine system design.
For how to set up or maintain the cluster, please check Installation and Management of TDengine Cluster.