Highlights
-
TSBS testing demonstrates that for the vast majority of queries, InfluxDB 3 Core returns results more slowly than InfluxDB 1.8, up to 48x slower in the DevOps use case and 30x slower in the IoT use case.
-
In the IoT use case, six of the twelve InfluxQL test queries could not be executed in Influx 3 Core due to incompatibility. These queries ran normally on InfluxDB 1.8, indicating that InfluxDB 3 Core’s support for InfluxQL is still limited.
-
When testing the larger TSBS dataset (4,000 devices with 10 metrics each), InfluxDB 3 Core could not perform at high concurrency and required reducing the number of workers to three or fewer.
Following our data ingestion testing of InfluxDB 3 Core published last week, we decided to compare the query performance of InfluxDB 3 Core with InfluxDB 1.8. Once again, we used the Time Series Benchmark Suite (TSBS) to ensure a level playing field for all database platforms. Because InfluxDB 3 Core supports InfluxQL, the TSBS query set for InfluxDB 1.8 should be able to run in the latest alpha without modification. However, during our testing, we found that InfluxQL compatibility is not complete.
InfluxData has listed in their documentation a number of InfluxQL features that are not yet supported in the latest alpha. Even taking these into consideration, we were unable to obtain results for several TSBS queries due to errors, whereas all queries return results normally in InfluxDB 1.8. Additionally, the queries that did return in InfluxDB 3 Core showed significantly lower performance compared with InfluxDB 1.8.
DevOps Use Case
As an initial performance comparison, we took the TSBS DevOps data from the smallest scenario of 100 devices and ran each query 4,000 times with 8 workers, obtaining the average response time.
| Query | InfluxDB OSS 3.0 | InfluxDB OSS 1.8 | TDengine OSS 3.3.5.8 | InfluxDB 3.0 vs. 1.8 | TDengine vs. InfluxDB 3.0 | TDengine vs. InfluxDB 1.8 |
|---|---|---|---|---|---|---|
| single-groupby-1-1-1 | 34.46 | 2.17 | 1.85 | 6.30% | 1862.70% | 117.30% |
| single-groupby-1-1-12 | 306.97 | 10.92 | 3.52 | 3.56% | 8720.74% | 310.23% |
| single-groupby-1-8-1 | 35.97 | 4.15 | 2.89 | 11.54% | 1244.64% | 143.60% |
| single-groupby-5-1-1 | 38.96 | 5.33 | 2.25 | 13.68% | 1731.56% | 236.89% |
| single-groupby-5-1-12 | 330.40 | 41.01 | 4.99 | 12.41% | 6621.24% | 821.84% |
| single-groupby-5-8-1 | 40.76 | 14.00 | 3.92 | 34.35% | 1039.80% | 357.14% |
| cpu-max-all-1 | 220.23 | 7.79 | 2.78 | 3.54% | 7921.94% | 280.22% |
| cpu-max-all-8 | 228.85 | 16.91 | 6.17 | 7.39% | 3709.08% | 274.07% |
| double-groupby-1 | 496.51 | 77.00 | 20.74 | 15.51% | 2393.97% | 371.26% |
| double-groupby-5 | 522.81 | 322.10 | 37.32 | 61.61% | 1400.88% | 863.08% |
| double-groupby-all | 556.91 | 614.01 | 55.63 | 110.25% | 1001.10% | 1103.74% |
| high-cpu-1 | 663.75 | 15.67 | 3.41 | 2.36% | 19464.81% | 459.53% |
| high-cpu-all | 947.92 | 1210.82 | 109.34 | 127.73% | 866.95% | 1107.39% |
| groupby-orderby-limit | 721.51 | 163.72 | 10.86 | 22.69% | 6643.74% | 1507.55% |
| lastpoint | 1905.56 | 39.65 | 10.71 | 2.08% | 17792.34% | 370.21% |
We can see that InfluxDB 3 Core outperformed InfluxDB 1.8 in only two categories, double-groupby-all and high-cpu-all. For the other queries, it ranged from 1.6x slower in double-groupby-5 to 48x slower in lastpoint.
Considering the cardinality improvements already evident in InfluxDB 3 Core, we increased the scale of the comparison from 100 devices to 4,000 devices to see whether the new version would perform better in a larger-scale scenario. However, at this point we began to encounter an error: InfluxDB began to reject HTTP requests, though it did not crash, and nothing was written to its logs. The error is shown as follows:
To resolve this issue, we had to reduce the number of workers to three or even two. Most of the more complex queries were affected: double-groupby-1, double-groupby-5, double-groupby-all, high-cpu-all, groupby-orderby-limit, and lastpoint.
IoT Use Case
We then moved to the TSBS IoT use case, which is more complex but more representative of the time-series data that TDengine users typically manage. Unfortunately, six of the twelve queries failed to run in InfluxDB 3 Core.
| Query | InfluxDB OSS 3.0 | InfluxDB OSS 1.8 | TDengine OSS 3.3.5.8 | InfluxDB 3.0 vs. 1.8 | TDengine vs. InfluxDB 3.0 | TDengine vs. InfluxDB 1.8 |
|---|---|---|---|---|---|---|
| last-loc | 370.27 | 12.02 | 2.30 | 3.25% | 16098.70% | 522.61% |
| low-fuel | 277.41 | 18.91 | 11.41 | 6.82% | 2431.29% | 165.73% |
| high-load | 352.29 | 12.28 | 2.36 | 3.49% | 14927.54% | 520.34% |
| stationary-trucks | — | 51.43 | 7.37 | — | — | 697.83% |
| long-driving-sessions | — | 11.94 | 9.22 | — | — | 129.50% |
| long-daily-sessions | — | 43.87 | 18.82 | — | — | 233.10% |
| avg-vs-projected-fuel-consumption | 997.92 | 262.05 | 101.38 | 26.26% | 984.34% | 258.48% |
| avg-daily-driving-duration | — | 259.20 | 82.80 | — | — | 313.04% |
| avg-daily-driving-session | — | 465.37 | 57.30 | — | — | 812.16% |
| avg-load | 3653.23 | 3435.50 | 12.28 | 94.04% | 29749.43% | 27976.38% |
| daily-activity | — | 135.41 | 63.11 | — | — | 214.56% |
| breakdown-frequency | 1245.37 | 1630.43 | 122.43 | 130.92% | 1017.21% | 1331.72% |
Note: Entries listed as — indicate that the query failed or returned no results.
InfluxDB 3 Core outperformed InfluxDB 1.8 in only one category, breakdown-frequency. For the other categories that returned results, it ranged from 1.1x slower in avg-load to 30x slower in last-loc.
The six queries that failed were stationary-trucks, long-driving-sessions, long-daily-sessions, avg-daily-driving-duration, avg-daily-driving-session, and daily-activity. Of these, the latter three queries were not compatible with the InfluxQL implementation in InfluxDB 3 Core. As an example, the avg-daily-driving-duration query returned the following error.
Similar errors occurred for avg-daily-driving-session and daily-activity. As an experiment, we modified the InfluxQL queries and were able to get them to run in InfluxDB 3 Core with some effort, demonstrating that the InfluxQL API is not fully compatible with InfluxDB 1.8.
In addition, the stationary-trucks, long-driving-sessions, and long-daily-sessions queries returned empty result sets. The queries all returned very quickly but did not contain any results. At this time, we have not been able to update these queries for InfluxDB 3 Core.
Furthermore, when we increased the size of the query dataset to 4,000 devices, we also had to reduce the number of workers as described in the DevOps scenario. The affected queries were last-loc, high-load, avg-vs-projected-fuel-consumption, avg-daily-driving-session, avg-load, and breakdown-frequency.
Try for Yourself
To automate the testing process and enable our users to compare databases using TSBS, we have developed a script that you can run on your own machine. On an Ubuntu 22 machine, clone the enh/add-influxdb3.0 branch of our TSBS fork to the /usr/local/src directory. Then open the scripts/tsdbComp directory and run the tsbs_test.sh --help command as the root user.
sudo -s
git clone https://github.com/taosdata/tsbs
cd tsbs
git checkout enh/add-influxdb3.0
cd scripts/tsdbComp
./tsbs_test.sh --helpThis describes the scenarios that you can test and the configuration options available to you. Note that for performance reasons, a machine with 4 cores and 8 GB of RAM is required to use this script.
Conclusion
TSBS testing shows that InfluxDB 3 Core is still inferior to InfluxDB 1.8 in terms of query performance. More importantly, InfluxQL compatibility is still an issue for InfluxDB 3 Core. In addition, as queries in InfluxDB 3 Core are limited to 72 hours, many of the typical time-series use cases, such as those in TSBS, will be difficult or impossible to implement. As InfluxDB 3 Core is still in the public alpha phase, we are sure that its performance and compatibility will be improved in later releases. However, we advise open-source users to be wary of upgrading for the time being.
