1. Executive Summary
This benchmark study demonstrates that TDengine achieves compression ratios 2.3x to 25.8x superior to InfluxDB across different real-world time-series datasets. These findings are supported by open-source test code available for independent verification.
2. Testing Methodology
To ensure complete transparency and reproducibility, all test code and procedures are publicly available on GitHub. The testing scenarios are taken from the independent and open-source TSBS framework.
2.1. Hardware Configuration
- CPU: Intel® Xeon® CPU E5-2650 v3 @ 2.30GHz (40 cores)
- Memory: 251 GB of DDR4 synchronous registered (buffered) RAM at 2133 MT/s
2.2. Software Configuration
- TDengine OSS 3.3.6.3, gitinfo
b6a63a76f552b4afb467eb970043471ffa8acfda - InfluxDB Core 3.0.0, revision
3b602eead2bb27aee74fb3cfc45f6be806d3b836 - Operating System: Ubuntu 22.04 LTS
2.3. Test Procedure
For each dataset, the disk space used by TDengine and InfluxDB was calculated as follows:
- Install and configure TDengine and InfluxDB.
- Generate and load the appropriate TSBS dataset into both TDengine and InfluxDB.
- Allow the database to complete any post-ingestion processes.
- Measure disk space usage using filesystem tools.
3. Test Results
The data generated by the TSBS IoT and DevOps use cases were used in this comparison.
3.1. IoT Use Case
The TSBS IoT use case models a fleet management scenario including separate data for diagnostics and readings for a group of trucks at a logistics enterprise. The dataset is described as follows:
- Diagnostics data: 3 metrics and 8 tags per truck with nanosecond-level timestamp
- Readings data: 7 metrics and 8 tags per truck with nanosecond-level timestamp
- 10 second collection interval
- Integer and floating-point metrics
- Integer and string tags
- Includes null and out-of-order data
The four scenarios for this use case are described in the following table:
| Scenario 1 | Scenario 2 | Scenario 3 | Scenario 4 | |
|---|---|---|---|---|
| Devices | 100 | 4,000 | 100,000 | 1 million |
| Duration | 2 days | 2 days | 3 hours | 3 minutes |
| Interval | 10 seconds | 10 seconds | 10 seconds | 10 seconds |
| Records per device | 15,549 | 15,558 | 972 | 16 |
| Total rows | 3,109,944 | 124,466,978 | 194,487,997 | 32,414,619 |
- Note that although Scenario 4 has the most devices, it has fewer total rows due to its short duration.
- Because null records can exist in this dataset, the number of records per device is an average.
3.2. Results
| InfluxDB Core | TDengine OSS | InfluxDB vs. TDengine | |
|---|---|---|---|
| 100 devices | 349 MB | 47 MB | 742.55% |
| 4,000 devices | 7424 MB | 1846 MB | 402.17% |
| 100,000 devices | 15929 MB | 3146 MB | 506.33% |
| 1 million devices | 3318 MB | 1423 MB | 233.17% |
Test results show that TDengine uses less disk space to store the data for all scenarios in the IoT use case. InfluxDB required between 2.3 and 7.4 times more space than TDengine to store the same data.
3.3. DevOps Use Case
The TSBS DevOps use case models an IT monitoring scenario for server CPUs in a data center. The dataset is described as follows:
- 10 metrics and 10 tags per CPU with nanosecond-level timestamp
- 10 second collection interval
- Integer metrics
- Integer and string tags
- Does not include null or out of order data
The four scenarios for this use case are described in the following table:
| Scenario 1 | Scenario 2 | Scenario 3 | Scenario 4 | |
|---|---|---|---|---|
| Devices | 100 | 4,000 | 100,000 | 1 million |
| Duration | 2 days | 2 days | 3 hours | 3 minutes |
| Interval | 10 seconds | 10 seconds | 10 seconds | 10 seconds |
| Records per device | 17,280 | 17,280 | 1,080 | 18 |
| Total rows | 1,728,000 | 69,120,000 | 108,000,000 | 18,000,000 |
- Note that although Scenario 4 has the most devices, it has fewer total rows due to its short duration.
3.4. Results
| InfluxDB Core | TDengine OSS | InfluxDB vs. TDengine | |
|---|---|---|---|
| 100 devices | 194 MB | 8 MB | 2425.00% |
| 4,000 devices | 7909 MB | 306 MB | 2584.64% |
| 100,000 devices | 7591 MB | 720 MB | 1054.31% |
| 1 million devices | 1858 MB | 706 MB | 263.17% |
In the DevOps use case, InfluxDB required between 2.6 and 25.8 times more space than TDengine to store the same data. Smaller-scale datasets were particularly well-compressed in TDengine, with InfluxDB using more than 20x the disk space of TDengine to store the same data from the 100- and 4,000-device datasets.
4. Why TDengine Achieves Superior Compression Ratios
- Two-stage compression: TDengine first encodes data based on its data type, for example delta encoding for timestamps and larger integers, ensuring highly efficient storage for time-series scenarios with similar data. LZ4 is then used to further compress the encoded data.
- “One table per device” design: TDengine creates a separate table for each device, ensuring that each block of data contains the records for a single table. This ensures that similar data is stored together and increases compressibility.
Learn more about TDengine’s high compression.
5. Reproducing These Results
We encourage you to verify these results and have developed a script with which you can run TSBS tests on your own machine. On an Ubuntu 22 machine, clone our TSBS fork to the /usr/local/src directory. Then open the scripts/tsdbComp directory and run the tsbs_test.sh --help command as the root user.
sudo -s
git clone https://github.com/taosdata/tsbs
cd tsbs
git checkout enh/add-influxdb3.0
cd scripts/tsdbComp
./tsbs_test.sh --helpThis describes the scenarios that you can test and the configuration options available to you.
Note that performance testing by nature requires machines with adequate hardware.
- If you would like to run the full test suite for the DevOps or IoT use case, use a server with at least 24 cores, 128 GB of RAM, and 500 GB of disk space.
- If you prefer to run the tests on a personal computer or smaller virtual machine, select the
cputestoriottestscenarios. These scenarios run a subset of TSBS that can return results within 45 minutes on most computers. For these scenarios, a machine with 4 cores, 8 GB of RAM, and 40 GB of disk space is required.
6. Business Impact Analysis
The compression ratio advantage translates directly to business value:
6.1. Storage Cost Savings
Although storage costs have decreased in recent years, TDengine’s high compression can still significantly reduce expenses, especially at scale. Consider Scenario 4 from the IoT use case described above. Over a month, a dataset of this size would generate over 48 TB of data as compressed by InfluxDB, compared with only 20.8 TB in TDengine. Assuming typical enterprise storage costs of $0.10/GB/month for hot storage, including backup and maintenance, this would result in a yearly reduction of $33,200 in storage costs.
6.2. Beyond Storage: Additional Benefits
While this study focuses solely on compression ratio, TDengine’s storage efficiency delivers additional benefits:
- I/O Performance Improvement: Reduced data volumes mean fewer I/O operations for the same logical data.
- Backup and Disaster Recovery Efficiency: A smaller data footprint reduces backup time and transfer costs.
- Network Transfer Reduction: For distributed systems and edge-cloud synchronization architectures, transferring compressed data can significantly reduce bandwidth requirements.
7. Conclusion
This benchmark study conclusively demonstrates TDengine’s superior compression ratio compared to InfluxDB across multiple real-world scenarios. With better compression ratios in all tested datasets, TDengine offers substantial advantages in storage efficiency, cost savings, and overall database performance.
The methodology employed in this study emphasizes transparency, reproducibility, and real-world applicability. By making all test code, data, and procedures publicly available, we invite independent verification of these results.
For organizations dealing with large volumes of time-series data, particularly those with high-cardinality requirements or hybrid edge-cloud deployments, TDengine’s compression advantage represents a significant technological and economic benefit.
