In IoT, Industrial IoT, and financial technology, storing and analyzing large volumes of time-series data has become a core challenge for enterprise digital transformation. As a database system purpose-built for time-series data, the distributed architecture of a TSDB directly determines its scalability, reliability, and performance. TDengine, an open-source high-performance time-series database, has become a widely adopted choice for organizations processing time-series data because of its distinctive distributed architecture design.
Design philosophy
Assume a single machine can fail
The TDengine distributed architecture is built on the principle that any single server can fail at any time. Every core component is engineered for high availability from the architecture level. TDengine maintains data integrity and service continuity even under hardware failures, network outages, and other abnormal conditions.
Horizontal scaling capability
One of the biggest challenges a time-series database faces is explosive data growth. Through its horizontally scalable architecture, TDengine lets users add physical nodes on demand to achieve linear expansion of both storage capacity and compute power. Whether dealing with millions or tens of billions of data points, TDengine handles the load by adding nodes.
Core logical units
The TDengine distributed architecture is made up of several logical units working together. Each unit has a specific role, forming a complete data processing system.
pnode (Physical Node)
A pnode (Physical Node) is the basic physical unit in a TDengine cluster, representing an independent computer or virtual machine. Each pnode is uniquely identified by its FQDN (Fully Qualified Domain Name) and is an independently deployable and manageable entity within the cluster.
<em>-- View physical nodes in the cluster</em>
SHOW DNODES;
dnode (Data Node)
A dnode (Data Node) is the taosd process instance running on a pnode. It is the core service process of TDengine. Each dnode contains multiple functional modules:
- vnode: Virtual Node, responsible for data storage
- mnode: Management Node, responsible for metadata management
- qnode: Query Node, responsible for query computation
- snode: Stream Node, responsible for stream processing tasks
A single dnode can host multiple types of nodes at the same time, enabling flexible resource scheduling.
vnode (Virtual Node)
A vnode (Virtual Node) is the most important data sharding unit in TDengine. Each vnode has its own threads, memory space, and storage directory, and is responsible for managing a specific time range of data shards.
Key characteristics of vnodes:
- Independent operation: Each vnode has its own write, query, and compaction threads
- Data sharding: Data is distributed across vnodes by time range
- Resource isolation: Resources between vnodes do not interfere with each other, preventing single points of bottleneck
<em>-- Create a database with vnode parameters</em>
CREATE DATABASE mydb VGROUPS 4 DURATION 10d;
mnode (Management Node)
An mnode (Management Node) handles metadata management for the entire cluster, including:
- User and privilege management
- Database and table schema information
- Cluster topology information
- Load balancing and scheduling
For high availability, mnodes use the Raft consensus protocol and support up to 3 mnodes forming a cluster. When the primary mnode fails, the system performs leader election to keep the metadata service running without interruption.
<em>-- Create mnodes</em>
CREATE MNODE ON DNODE 2;
CREATE MNODE ON DNODE 3;
qnode (Query Node)
A qnode (Query Node) is a compute node introduced in TDengine 3.0 that enables storage-compute separation. When a query requires substantial compute resources, the system distributes the computation across multiple qnodes for parallel execution, improving performance for complex queries.
Advantages of qnodes:
- Storage-compute separation: Compute resources can be scaled independently
- Load balancing: Query tasks are evenly distributed across qnodes
- Elastic scaling: The number of qnodes can be adjusted dynamically based on query load
snode (Stream Node)
An snode (Stream Node) is dedicated to stream processing tasks. TDengine can trigger computation logic in real time as data is written, and snodes are responsible for executing these stream processing tasks to deliver real-time data analysis and processing.
vgroup (Virtual Node Group)
A vgroup (Virtual Node Group) is a logical group of multiple vnodes used to implement multi-replica data storage. Vnodes within each vgroup maintain consistency through the Raft consensus protocol, ensuring data reliability and high availability.
<em>-- Create a database with 3 replicas</em>
CREATE DATABASE mydb REPLICA 3;
Node communication
Nodes in a TDengine cluster communicate over TCP, with the following capabilities:
Data compression
To reduce network transmission overhead, TDengine supports compressing data during node-to-node transfers. Users can choose an appropriate compression algorithm based on their network environment and data characteristics.
Digital signatures
For communication security, TDengine supports digital signatures on messages transmitted between nodes to prevent tampering.
Connection management
TDengine uses persistent connections between nodes to avoid the overhead of repeated connection setup. The system also periodically checks connection health to detect and handle network anomalies promptly.
Data write flow: a complete process
A typical data write flow shows how the components work together.
The 8 steps of a write operation
- Client connection: The application connects to the cluster through the taosc client library and retrieves the mnode address.
- Metadata query: The client queries the mnode for the vgroup information of the target table.
- Route retrieval: The mnode returns the location information of each vnode in the vgroup.
- Connection establishment: The client establishes a connection to the dnode where the target vnode resides.
- Data transmission: The client sends the write request to the primary vnode (leader).
- Raft replication: The primary vnode synchronizes the data to the follower vnodes through the Raft consensus protocol.
- Acknowledgment: Once a majority of replicas confirm the write, the primary vnode returns a success response to the client.
- Cache flush: Data is first written to the memory cache and then asynchronously flushed to disk.
<em>// Typical data write code example</em>
taos_init();
TAOS *taos = taos_connect("localhost", "root", "taosdata", NULL, 0);
TAOS_STMT *stmt = taos_stmt_init(taos);
<em>// Prepare the insert statement</em>
taos_stmt_prepare(stmt, "INSERT INTO ? VALUES (?, ?)", 0);
<em>// Bind parameters and execute</em>
TAOS_BIND params[2];
<em>// ... set parameter values</em>
taos_stmt_bind_param(stmt, params);
taos_stmt_add_batch(stmt);
taos_stmt_execute(stmt);
High availability guarantee
In the above flow, if the primary vnode fails, the Raft protocol triggers a leader election. A new primary vnode is selected from the remaining replicas, ensuring that the write service continues without interruption. The entire process is transparent to the client and requires no manual intervention.
Distributed query optimization
The TDengine distributed architecture supports efficient data writes and applies deep optimization for time-series data queries.
Partition pruning
When executing a query, the system first determines which vnodes need to be accessed based on the time range condition, avoiding scans of irrelevant data partitions.
Parallel querying
For queries that span multiple vnodes, TDengine distributes the query tasks to each vnode for parallel execution and then aggregates the results.
Computation pushdown
For aggregation queries, TDengine pushes computation logic down to the vnode layer. Only the aggregated results are returned, reducing network transfer volume.
<em>-- Query example: using distributed computing capability</em>
SELECT AVG(temperature), MAX(humidity)
FROM sensor_data
WHERE ts > NOW() - 1h
PARTITION BY location;
Summary
TDengine is a distributed database designed specifically for time-series workloads. Its architecture accounts for the characteristics of time-series data and the demands of enterprise applications. The design starts from the assumption that any single machine can fail, builds on the coordinated operation of vnodes, mnodes, qnodes, and snodes, and uses the Raft consensus protocol to provide high availability. The result is a complete, reliable, high-performance distributed data processing system.
Whether for real-time monitoring of IoT devices, data collection in industrial systems, or time-series analysis of financial transactions, TDengine provides stable and efficient data services. Its horizontal scaling capability helps organizations handle data growth, while its storage-compute separation architecture provides flexible compute resource scheduling for complex analytical workloads.
For organizations and developers evaluating time-series database options, the TDengine distributed architecture is worth a close look. With sound cluster planning and configuration, TDengine helps users build a time-series data processing platform that fits their requirements.
