TDengine TSDB taosX Zero-Code Data Ingestion Guide

In industrial IoT, connected cars, and energy management scenarios, time-series data collection and ingestion is often the first barrier to project delivery. Traditional approaches require writing substantial amounts of code. Development cycles stretch long and stability issues creep in easily. TDengine Enterprise Edition addresses this with taosX, a component that enables fast multi-source data ingestion with zero coding.

What is taosX

taosX is the core data ingestion component of TDengine Enterprise Edition. It provides two operating modes: service mode, where users operate through the taosExplorer graphical interface for those less familiar with the command line, and command-line mode, where synchronization tasks run via command-line arguments for automated operations.

Supported data source types

taosX supports a wide range of data sources:

• taos: Pull data from another TDengine instance
• tmq: Data Subscription (TMQ) for TDengine data change capture
• kafka: Read messages from Kafka
• influxdb: Migrate data from InfluxDB
• pi: Pull data from AVEVA PI System
• opc: Connect via OPC UA and OPC DA protocols
• mqtt: Subscribe to MQTT brokers
• csv/parquet: Batch import from files

Four typical use cases

TDengine version migration. Move data from TDengine 2.x to 3.x. taosX handles schema differences and ensures data is transferred reliably.

Heterogeneous database import. Import data from PI System, InfluxDB, OPC, and other sources. taosX automatically handles data type mapping and format conversion, removing the need for manual transformation scripts.

Historical file batch import. Load CSV or Parquet files containing historical time-series records. The import path supports configurable delimiters, timestamp columns, and table mappings.

Real-time data synchronization. Stream data continuously from Kafka, MQTT, and other message queues. Service mode supports both real-time and scheduled sync strategies.

Service mode walkthrough

Start the taosX service with taosx server start. In the taosExplorer web interface, click the “Data Ingestion” menu, select a data source type, fill in the connection parameters, configure data mapping relationships, and set the sync strategy (real-time or scheduled). The entire process requires no coding.

This mode is designed for teams who prefer visual configuration. A typical setup takes a few minutes rather than the days or weeks a hand-coded integration would demand.

Command-line mode walkthrough

The basic syntax is:

taosx sync --source <source DSN> --target <target DSN> [options]

Example 1: Ingest from Kafka

taosx sync \
  --source "kafka://broker:9092/topic=metrics&group=consumer-group" \
  --target "taos://user:pass@hostname:6030" \
  --batch-size 5000 \
  --workers 4

The Kafka source DSN specifies the broker address, Topic, and Consumer Group. The --batch-size and --workers options control write throughput.

Example 2: Import from CSV files

taosx import \
  --source "/data/sensor_readings.csv" \
  --target "taos://user:pass@hostname:6030" \
  --table "sensors" \
  --timestamp-column "ts" \
  --delimiter ","

The import subcommand is designed for file-based batch loading. The --table option maps data to the target table, --timestamp-column identifies the primary time column, and --delimiter specifies the field separator.

Example 3: Migrate from InfluxDB

taosx sync \
  --source "influxdb://user:pass@influxdb-host:8086/database=mydb" \
  --target "taos://user:pass@tdengine-host:6030" \
  --measurement "cpu"

When migrating from InfluxDB, taosX reads the specified Measurement and writes it to the TDengine TSDB target. Type mapping and schema conversion happen automatically.

Data mapping configuration

Mapping rules are defined in a JSON configuration file. Three capabilities are supported:

Field mapping. Define how source fields map to target columns. For example, a source tag maps to a target column name.

{
  "field_mappings": [
    {"source": "source_tag", "target": "target_column"}
  ]
}

Filter conditions. Include or exclude records based on field values.

{
  "filters": [
    {"column": "status", "operator": "=", "value": "active"}
  ]
}

Transformation functions. Apply conversion logic during ingestion.

{
  "transformations": [
    {"column": "temp_value", "function": "celsius_to_fahrenheit"}
  ]
}

Performance tuning recommendations

• Set --batch-size between 1000 and 10000. Larger batches increase throughput but consume more memory. Start at 5000 and adjust based on your data size.
• Increase --workers to add concurrency. More worker threads parallelize the ingestion pipeline. Watch CPU usage to find the sweet spot.
• Use --compress to reduce network transfer volume. This is especially useful when the data source and target are on separate networks.
• For large files, use --partition to break the dataset into smaller segments. Partitioned processing improves parallelism and reduces the impact of any single failure.

Summary

taosX lowers the technical barrier to time-series data ingestion with its zero-code, GUI-driven, multi-source approach. For teams running TDengine Enterprise Edition, taosX is the recommended first choice for data onboarding, whether migrating from a legacy database, importing historical files, or setting up real-time data pipelines.