High-Performance TSDB Writes with Connector-Based Batching

Large-scale data collection scenarios put heavy write pressure on a time-series database. Sensor ingestion in industrial IoT and metric reporting in IT operations can reach very high write volumes, and teams need to raise throughput without making application code difficult to maintain. Connector-based batching and asynchronous writes are one way to reduce that complexity.

High-performance write architecture

In high-performance write mode, the TDengine TSDB connector automatically creates write threads and dedicated queues. The core mechanism splits and caches data by Subtable, then sends it to the server in batches when a row-count threshold or timeout condition is reached. This design reduces network round trips and raises overall throughput.

For developers, the biggest advantage is achieving high-performance writes without needing to master multi-threaded programming or data partitioning techniques. The connector handles all complex data organization in the background, including subtable routing, data sorting, and batch packaging, keeping application code clean. Teams can focus on business logic rather than low-level write optimization.

JDBC high-performance write capabilities

In the Java ecosystem, the time-series database JDBC connector provides comprehensive high-performance write support. Key capabilities include:

• Standard Parameter Binding interface: supports the JDBC-standard Parameter Binding approach, so developers do not need to learn a new API
• Linear scalability: write throughput scales linearly with the number of configured write threads, allowing flexible adjustment for business needs
• Fault tolerance: supports configurable retry count and retry interval after write timeouts or connection drops
• Write statistics: the executeUpdate interface returns the number of written rows, supporting monitoring and debugging
• Resource isolation: write threads are separated from business threads, preventing blocking of the main application flow

Key configuration parameters

To get the most out of high-performance writes, configure the following parameters carefully:

• PROPERTY_KEY_BACKEND_WRITE_THREAD_NUM (default 10): the number of write threads, which directly determines write parallelism
• <strong>PROPERTY_KEY_BATCH_SIZE_BY_ROW</strong> (default 1000): batch size per submission
• PROPERTY_KEY_CACHE_SIZE_BY_ROW (default 10000): internal cache size
• <strong>PROPERTY_KEY_ENABLE_AUTO_RECONNECT</strong> (default false): whether to enable automatic reconnection
• <strong>PROPERTY_KEY_RECONNECT_INTERVAL_MS</strong> (default 2000 ms): reconnection interval
• PROPERTY_KEY_RECONNECT_RETRY_COUNT (default 3): maximum reconnection retries

Batch size and cache size should be balanced against per-row data size and available network bandwidth. A batch size that is too small increases request count; one that is too large may consume excessive memory. In high-performance write mode, set auto-reconnect to true to handle network instability.

Enabling high-performance write mode

Enabling high-performance write mode takes two steps:

First, set <strong>PROPERTY_KEY_ASYNC_WRITE</strong> to <strong>stmt</strong> in the connection properties, or add asyncWrite=stmt to the JDBC URL.

Second, use ASYNC_INSERT INTO instead of the traditional <strong>INSERT INTO</strong> in the SQL used to create the PreparedStatement.

Note: ASYNC_INSERT INTO changes only the data submission path. The SQL syntax is fully consistent with INSERT INTO and supports all standard write syntax, including multi-table writes and automatic table creation with Tags.

Typical application scenarios

High-performance write features are especially suited to scenarios where a client program continuously reads data from other sources using multiple reader threads, with backend write threads matching them. Each write thread has its own fixed-size message queue, ensuring data written to the time-series database remains ordered and is not lost.

For example, pulling data from Kafka and writing it to a time-series database is a typical pattern. In this architecture, Kafka consumer threads fetch data while write threads handle batch submission. The two are decoupled through queues, maintaining consumption speed while achieving efficient writes. In production, match the Kafka consumer thread count to the write thread count, or keep them proportional, to avoid a bottleneck at either stage. You can also combine Kafka consumer groups with horizontal scaling of multiple application instances to further raise overall write throughput.

Summary

The high-performance write connector in a time-series database lowers the barrier to high-throughput data ingestion through automated thread management, queue buffering, and batch submission. With proper configuration of thread count, batch size, and cache parameters, it can support million-point-scale write workloads. In practice, start with the default parameters and run benchmarks, then tune gradually based on workload characteristics to avoid the uncertainty of premature optimization. TDengine TSDB provides this connector-based high-performance write path for teams building large-scale ingestion pipelines.