In industrial data projects, ingestion is often where schedules slip. Each source brings its own protocol, format, timestamp behavior, and mapping rules. TDengine Enterprise reduces that work with browser-based, zero-code ingestion: teams configure source connections, parsing, and mappings instead of building custom pipelines for every integration.
The core tool: taosExplorer
TDengine Enterprise includes taosExplorer, a visual data management tool that serves as the unified data ingestion entry point. Users complete four steps:
- Select the target data source type
- Configure connection parameters
- Define data mapping and transformation rules
- Start the data ingestion task
This approach lowers the barrier for operations engineers and data analysts, while still giving developers a repeatable way to manage ingestion rules.
17 supported data sources
The solution covers industrial protocols, message queues, relational databases, time-series databases, and files.
Industrial data sources:
- AVEVA PI System
- AVEVA Historian
- OPC DA
- OPC UA
Direct connectivity can reduce the need for extra middleware in common industrial integration scenarios.
Message queues:
- MQTT
- Kafka
Databases:
- InfluxDB
- OpenTSDB
- MySQL
- Oracle
- PostgreSQL
- SQL Server
- MongoDB
Files and other:
- CSV
- TDengine Query (cross-instance query and write)
- TDengine Data Subscription (subscription-based sync)
Built-in ETL capabilities
Step 1: Parse. Raw data, especially from MQTT, often arrives as JSON strings. Supported parsing methods: JSON Path, regular expressions, and UDT custom scripts.
Step 2: Extract / Split. A single raw field may encode multiple dimensions. Supports split by delimiter and regex capture groups.
Step 3: Filter. Conditional expressions determine which messages get written. Capabilities include comparison operators, string functions (contains, matches, prefix/suffix), and compound expressions with AND, OR, NOT.
Step 4: Transform and Mapping. Maps parsed fields to supertable columns.
| Mapping rule | Description | Typical scenario |
|---|---|---|
| mapping | Direct mapping | Direct field-to-column mapping |
| value | Constant value | Fixed values (e.g., data source label) |
| generator | Timestamp generator | Auto-generate or convert timestamps |
| join | String concatenation | Concatenate multiple fields |
| format | String formatting | Output by template format |
| sum | Numeric addition | Sum multiple numeric fields |
| expr | Math expression | Arithmetic operations |
Practical value
- Lower technical barrier: Business and ops staff need no coding skills
- Faster implementation: Configuration-based approach shortens deployment cycles
- Simplified operations: Visual rule management aids troubleshooting
- Greater system flexibility: Modifying mapping rules requires only configuration changes, not redevelopment
Summary
With taosExplorer and built-in ETL, TDengine Enterprise supports configurable ingestion across 17 source types. For real-time collection, historical migration, and cross-system synchronization, this reduces custom code while keeping parsing, filtering, and transformation rules visible and maintainable.
