Connected cars constantly generate large quantities of time-series data – from metrics such as speed and position to timestamped events such as collisions. And for storing and processing smart vehicle data, fast writes, fast queries, and high compression are essential. However, many enterprises continue to use general-purpose databases such as MongoDB or Apache HBase for their data. These databases are more well-known to technical personnel, and when enterprises are just getting started – before the scale of their data expands – they can do the job.
Unfortunately, as the number of monitored devices grows and queries become more complex, general purpose solutions reveal their weaknesses in connected cars scenarios – write speed is too slow to support business growth, and enterprises are forced to scale out their database resources, resulting in excessive cost of operation. In this article, three cases are introduced in which TDengine reduced costs and improved performance for connected car data processing.
TDengine @ DJI Automotive
"In DJI Automotive's current cloud platform, the application of TDengine not only saves storage costs and development and learning costs, but also shows good write and read performance, meeting the processing needs of massive time series data on the intelligent driving cloud platform. On the query side, whether it's a query that selects specific data or a lightweight query, it returns data in milliseconds."
Background
DJI Automotive, a subsidiary of industry-leading drone manufacturer DJI, announced its entrance into the field of autonomous driving last April. Considering the high frequency at which its smart cars report data back to the system, DJI Automotive quickly realized that a time-series database (TSDB) management system would be necessary to process this massive set of autonomous driving data.
In addition to meeting DJI's performance criteria, TDengine also provides strong compression capabilities, enables high concurrency, and reduces O&M costs. It is open-source software and even its community edition supports clustering. Finally, the TDengine concept of “one table per device” provides a database structure that is particularly suited to DJI Automotive – the data for each vehicle is stored in a separate table. In the end, DJI Automotive chose TDengine to process its time-series data.
Architecture
TDengine @ NIO
"In practice, after the migration was completed, the query speed is significantly improved: HBase returned the 24-hour data of a single device in seconds, whereas TDengine returns the same data in milliseconds, and the storage space occupied by incremental data every day is the same. By implementing TDengine, the cost of cluster computing resources has been reduced by more than 60% compared with HBase."
Background
NIO Power has invested heavily in EV charging infrastructure in order to provide customers with a faster and convenient charging experience. All this charging infrastructure requires equipment monitoring and analysis. Data is collected by the equipment and transmitted to the cloud for storage so that the business operation team has both a real-time as well as historical view of the infrastructure. Originally, the system consisted of MySQL + HBase for data storage. MySQL stores the latest real-time data from the devices, and HBase stores the historical data. With the rapid deployment of equipment in swapping stations and supercharging stations across the country, the amount of equipment has grown substantially and the amount of accumulated data is increasing exponentially.
Architecture
TDengine @ Li Auto
"TDengine's functions exceed expectations, and its performance is much stronger than InfluxDB. The two-level storage architecture design (row storage and column storage) is great, and the TTL and labeling mechanism is transparent to the business. It has excellent high concurrent writing and data compression capabilities, which greatly reduces business costs and business pressure. Therefore, we decided to migrate from TiDB to TDengine."
Background
Li Auto has a typical high-concurrency insert scenario, with more writes and fewer reads. Currently there are 70,000 writes per second, and they expect to reach more than 200,000 writes per second in the next 3 years. Their previous system used MongoDB but was migrated to TiDB to facilitate expansion. However, for time series data, the cost-effectiveness of writing to TiDB is very low. In addition, it imposes unnecessary complexity. The underlying database table must be built for each month, and each collection point must be labeled. It is also not optimal or even suitable to perform large batch writes. In general, the current architecture was unable to meet their business requirements.
Li Auto then decided to try the time series database TDengine. The performance and functionality of TDengine exceeded our expectations when we performed in-depth technical and business tests. TDengine’s excellent high concurrency writing and data compression capabilities greatly reduce business costs and it’s native clustering features ensure scalability. With all of this in mind, we decided to migrate from TiDB to TDengine.
Architecture
