When I open-sourced the core components of TDengine back in 2019, I had no idea that within four years it would become one of the top time-series databases in the world, with more than 350,000 installations in over 50 countries. At the time, in fact, many of my peers questioned whether I would be able to turn a profit on open-source software — after all, who would pay money for a product that they could download for free? But by opening my source code, I also opened the door to unprecedented levels of community engagement and growth.
Four years ago, I made the right decision to open-source TDengine. And now I have made a new decision — to build on the open source TDengine OSS and create a next generation data historian for traditional industries that is easy to start and affordable to all with no upfront costs.
Creating a Zero-Code Industrial Data Platform
For traditional industries, which typically are not able to maintain large IT departments and instead rely on systems integrators, deploying and using new data systems can be a challenge. However, the trend toward digital transformation and smart manufacturing has led to demand among traditional industries for modern data storage and analysis.
Not having a background in industry myself, I visited a number of sites — power stations, oilfields, chemical plants, mines, and more — and was amazed by the scale of their operations. But equally amazing was that their data infrastructure of these modern plants seemed to be decades old, with outdated user interfaces still running on ancient Windows systems.
And more importantly, these data systems were exorbitantly priced, typically based on the number of data points. At several dollars per point, the data infrastructure for a power plant with tens of thousands of points could easily end up costing six figures. Not only that — for these plants, simply adding data points required renegotiating their contract with the vendor. In today’s interconnected world, where everything is generating more and more data every day, it was unthinkable to me that this pricing model could still be in use.
But escaping from these systems was no easy task; they were closed systems that were difficult to integrate with third-party tools, if integration was even possible. With the advent of big data and artificial intelligence, an increasing number of industrial enterprises are hoping to introduce AI-enabled analytics into their workflows, only to find themselves unable to integrate their data infrastructure with AI platforms due either to technical or cost challenges.
In addition, as enterprises go through mergers and acquisitions, they can easily end up with multiple production lines and plants running different industrial data platforms whose data cannot be easily centralized and managed. With the systems currently at place in many industries, simple but important tasks such as data cleaning, transformation, and governance all become uphill battles.
Over the past decade, adoption of Industrial IoT (IIoT) and big data technologies has become increasingly widespread among traditional industries, but the creation of IIoT platforms from scratch is not generally feasible for industrial enterprises, which cannot hire large development teams to build systems that will provide a return on investment years in the future if at all. These challenges have caused industry leadership to be less than optimistic about the prospects of AI and Industry 4.0.
Developing the Next Generation of Industrial Data Software
After learning about traditional industrial data systems and the pain points of their customers, I was inspired to create a new platform for industrial data that would be easy to get started with and affordable and accessible to all with no upfront costs to deploy. That was two years ago, when the TDengine Team first started working on integration with industrial data sources. Now, we are proud to announce our zero-code solution for continuously ingesting industrial data, including OPC-UA, OPC-DA, and MQTT, in real time.
With only a simple configuration process, enterprises can ensure the quality of their data through custom rules for data cleaning and transformation. Our solution eliminates the need to deploy separate ETL or stream processing tools, further reducing system complexity and operational expenses. In addition, the standard JDBC and ODBC interfaces included with TDengine enable seamless integration with a variety of BI and visualization tools. With zero-code support for industrial data processing, TDengine is now more than just a time series database — it is a next generation data historian.
Even considering the advantages TDengine has to offer, adopting a new data system can be difficult for enterprises. Traditional industries without dedicated DevOps teams may struggle to complete the deployment, configuration, and optimization process. In these situations, TDengine Cloud comes to the rescue. TDengine Cloud delivers our data historian as a fully managed cloud service, allowing enterprises to ingest data from their PLCs and other devices and then visualize and analyze the data in Grafana, Power BI, or other tools.
Anyone who registers for TDengine Cloud gets a free month of our starter plan so that you can test our system at no cost and without installing anything on your local systems. After the trial period ends, you can continue using the starter package — with enough resources to process data from 50,000 tags collected by PLCs — for only $1640 a month, making it a much more affordable data solution than traditional historians.
Breaking Up the Industrial Data Monopoly
In the battle against traditional industrial software, with its closed ecosystems, outdated architectures, prohibitive pricing, and vendor lock-in, there are two key weapons waiting to be brought into action: open source and the cloud.
Open source software prevents vendor lock-in by introducing an open ecosystem. This is a stark contrast to existing industrial platforms, whose interfaces are proprietary and often undocumented — even something as simple as a new reporting tool can be integrated only with assistance and permission from the vendor. Open systems, however, can connect with a wide variety of analytics, reporting, and AI products through standardized and open interfaces. Not a single traditional data historian is an open-source product. This is an opportunity for TDengine.
An often overlooked element of an open ecosystem is that it allows developers to focus on what they are best at, knowing that additional components or utilities will be provided by experts in those fields. TDengine does not provide visualization, analytics, or reporting components, choosing instead to focus all of our efforts on data ingestion, querying, and compression. Instead, TDengine enables seamless integration with a wide variety of third-party software, allowing enterprises to select their preferred options for these features. Furthermore, TDengine’s data subscription component can export data from TDengine into other data management systems, ensuring that you are never locked in to TDengine and can easily migrate if needed.
In most cases, cloud services are clearly beneficial for industrial scenarios — enterprises can control usage and costs without needing to deploy any hardware or software components on their own. Network security is offloaded to the cloud service provider so that enterprises without technical experts can avoid unintentional data disclosure and vulnerabilities. All in all, cloud services allow enterprises to quickly validate whether new systems meet their expectations without undergoing multiple rounds of sales meetings, taking on unneeded risk, or incurring fees.
A Bright Future Ahead
I believe it is inevitable that open source and the cloud disrupt the industrial software sector. Before long, a new generation of industrial software will revitalize the market, replacing everything from SCADA systems to industrial design software. In this new landscape, TDengine, a next generation data historian built on a cloud native, open source core, is perfectly positioned to take the place of legacy data historians.
Considering how the market has developed, it’s clear that my decision to go open source four years ago was right. Now, I am more confident than ever that our focus on industrial data is equally correct and that TDengine will play an important role in accelerating digital transformation and making cutting-edge technologies like big data and AI affordable and accessible to industrial enterprises of all sizes.
If you are interested in industrial data processing, sign up for a free month of TDengine Cloud today — no credit card required — and see for yourself how easy it is to get started. After trying our cloud service to verify that the solution is a good fit for your company, you can continue using TDengine Cloud or contact our sales team to learn about TDengine Enterprise, our solution for private deployments. You are also welcome to download the free, open source TDengine OSS or view our source code on GitHub. If you have any comments on our work, I would love to hear from you — your encouragement and criticism are our best source of motivation in our quest to democratize industrial data and make big data accessible to everyone.
