How TDengine centralizes data from various sources to streamline data analytics and sharing
In the age of IIoT, effective data management, including accessibility, is critical. The proliferation of connected IIoT sensors has generated a massive volume of time-series data that needs to be shared, centralized, and analyzed. This article discusses the value of modernizing time-series data infrastructure and the challenges associated with it. Specifically, it explores the capabilities and limitations of PI System and illustrates how the time-series database TDengine can be used to extend this system to enable data centralizing and sharing.
Managing & Sharing Data in the IoT Era
The Proliferation of IIoT Data
Enabled by the Internet of Things (IoT) and the digital transformation of traditional industries, the Industrial Internet of Things (IIoT) is the greatest advancement in industry since the electrification of factories in the early 20th century.
In practice, the IIoT is a connected network of devices and sensors that automatically monitor equipment and processes in real time, vastly improving efficiency, reducing downtime, and supporting advanced business intelligence. Naturally, this approach generates a massive volume of time-series data that needs to be collected and analyzed.
With the number of connected sensors increasing exponentially, it is critical for businesses to implement effective strategies, processes, and technologies to ingest and manage this data at scale.
Benefits of Centralizing IIoT Data
With the number of connected sensors increasing exponentially, it is critical for businesses to implement effective strategies, processes, and technologies to ingest and manage this data at scale. In addition to collecting and transforming operational data, it is also essential that data from individual sites be centralized and shared. In this way, organizations can benefit from a global view of the entire network of connected sensors and devices and develop insights that lead to company-wide efficiencies.
-
Unified worldview:
-
Instead of analyzing fragmented, siloed data, centralization allows users to view data from all sites on a single dashboard.
-
Users can review global operations while retaining the ability to zero in on individual sites.
-
-
Advanced analytics:
-
TDengine provides basic analytics via standard SQL with specialized extensions for time-series data, while also enabling real-time analytics.
-
With its open interfaces, you can also integrate TDengine with industry-leading analytics and visualization tools such as Seeq and Power BI.
-
-
Controlled sharing:
-
Data granularity can be controlled throughout the organization, enabling the sharing of entire datasets or limited segments.
-
Data segments can be shared with external partners, vendors, or regulators as needed with fine-grained privilege management.
-
PI System and the Challenges of the IIoT
Many industrial enterprises, especially in traditional industries such as energy and manufacturing, have developed or acquired unique data collection and operation systems within many of their sites.
AVEVA PI System, a platform that collects and stores time-series data, has been widely adopted in traditional industries to monitor and report on individual sites. To ensure reliability and low latency, a PI Server instance is deployed in each site, often physically close to the data that it collects. This architecture often requires a significant investment in software, hardware, and time for deployment and administration.
Although PI System excels at collecting data for individual sites, it has significant limitations when it comes to centralizing data from multiple sources. It usually requires either complicated IT implementations or exporting and categorizing vast amounts of information manually, which holds back real-time analytics.
In addition, PI System continues to use a pricing model based on traditional licensing, charging customers for the number of tags in use on their systems. This traditional pricing model is not a good fit for IIoT deployments and can quickly result in high costs as datasets increase in size.
In some cases, the limitations of PI System have resulted in further fragmentation of a business’s data stack, as rather than finding a solution that supplements PI System’s capabilities, new solutions are brought in to manage each scenario that PI cannot handle.
AVEVA CONNECT Data Services (Data Hub)
Shortly after acquiring OSIsoft in 2020, AVEVA launched AVEVA Data Hub (later rebranded to AVEVA CONNECT data services) to facilitate sharing across PI System deployments. In theory, this cloud-based system removes many of the barriers to sharing data from disparate PI Servers, providing secure access to real-time data for users inside or outside of the company’s network. Bidirectional sharing is also available for a number of services within the AVEVA ecosystem.
This service isn’t an accessible extension of the PI database, however, as it operates largely independently — Data Services is built on a separate data structure and lacks PI’s widely-used Asset Framework (AF). The cost and limitations on the number of data points that can be shared are also a drawback for some users. Sharing data from 3,000 streams costs mid-five figures annually, which adds up to an extremely hefty outlay for companies with tens of thousands, or millions of metrics to access.
Modernizing the Industrial Data Infrastructure
TDengine is a high-performance time-series database purpose-built for Industry 4.0 and Industrial IoT. TDengine can support billions of IoT devices effortlessly while consistently outperforming other databases in data ingestion, querying, and compression. And TDengine allows fast data replication to the cloud, reducing deployment time from months to minutes.
Even with these impressive capabilities, TDengine is user-friendly and intuitive. On the surface, it looks like a relational database and uses standard SQL; however, it’s designed as a real-time data engine optimized for data in flight. TDengine’s unique architecture also makes it easy to share and replicate data across multiple instances, enabling rapid access to vital data.
With TDengine’s data connectors, you can ingest data from PI System into TDengine, enabling seamless sharing and centralization of data coming from PI Servers on the edge. Instead of manually uploading files or using custom-built connectors, you can stream data continuously from PI Points or AF templates to TDengine Cloud. After a zero-code configuration process, TDengine connects with your PI System; handles extract, transform, and load (ETL); and enables automatic backfill after any disruptions.
With your data stored efficiently and securely in TDengine, you have access to many new possibilities. The standard, open interfaces that TDengine provides offer integration with analytics, visualization, and business intelligence (BI) tools so that you can do the data analysis that you want. Centralizing your operational data in TDengine can help you break down data silos and generate actionable business insights.
Architecture
The TDengine for PI System solution uses the AF SDK to query historical data from PI Data Archive, set up PI and AF data pipes for streaming data, and connect to PI AF to query the AF structure. It also creates the required tables and writes this data over a secure RESTful API to TDengine Cloud.
The TDengine Data Reference is an AF Custom Data Reference that queries data within TDengine and allows users to interact with TDengine as though it were a PI Point within the PI Data Archive. In this way, TDengine data can be used alongside PI data in PI Vision.
The Open System Future
Built on an open-source core, TDengine offers an open ecosystem aligned with the future of IIoT. You can ingest data from a variety of industrial sources into TDengine, including OPC and MQTT data in addition to data from PI System. With your data centralized in TDengine, you can then integrate TDengine seamlessly with modern analytics and visualization tools such as Seeq or Power BI and perform analytics on large-scale datasets. The open nature of TDengine makes data sharing an easy task and enables endless opportunities to generate useful insights that drive greater efficiencies and improve outcomes.
Conclusion
Instead of manually uploaded files and unmaintainable custom code, TDengine offers a solution that automates data centralization, gets data to the cloud while maintaining its context, and integrates with industry-leading analytics and BI tools such as Seeq and Power BI. And unlike traditional industrial data platforms, which lock customers into closed systems that leave them at the mercy of vendors when it comes to pricing and third-party integration, TDengine makes big data accessible and affordable to everyone.
With TDengine, organizations can continue to use existing edge servers while streamlining their analytics processes. TDengine enables efficient centralization and sharing of data collected from any number of industrial data sources along with data cleaning and ETL, so that you can make the most of your operational data. And with its easy-to-use interface and predictable pricing model suited for the IIoT, including a free trial for new users, TDengine can be a powerful and cost-effective extension of PI System for industrial enterprises.
