Why schemaless writing matters
IoT applications ingest large volumes of data points for automated management, business analysis, and device monitoring. The problem is that collection requirements change. Application logic gets updated. Hardware gets swapped. Each time, the data schema may need to change too.
With a traditional database, every schema change means running a DDL statement by hand. When you have thousands of devices, this manual approach gets slow and mistakes creep in.
A time-series database with schemaless writing avoids this. Developers do not need to pre-create supertables or subtables. The database builds the storage structure automatically from the data as it arrives.
How schemaless writing works
The idea is simple: the data drives the structure. During ingestion, the database handles four tasks on its own:
- Auto-creates supertables: If the target supertable does not exist, the database creates it from the incoming data.
- Auto-creates subtables: If the target subtable does not exist, the database creates one.
- Auto-adds columns: When incoming data includes a new column, the database adds it. Columns are append-only; they cannot be removed through the schemaless path.
- Auto-extends column width: When BINARY or NCHAR data exceeds the current column limit, the database widens the column.
The result: developers focus on application logic, not on table structure maintenance.
Line protocol format
TDengine TSDB’s schemaless writing works with three protocols: the InfluxDB line protocol, the OpenTSDB Telnet-style line protocol, and the OpenTSDB JSON format protocol. The InfluxDB line protocol is the most widely used:
measurement,tag_set field_set timestamp
Each component:
- measurement: The table name, which maps to a subtable in TDengine.
- tag_set: Tag data that identifies the data source.
- field_set: Regular data columns.
- timestamp: The primary key timestamp.
Here is an example:
meters,location=California.SanFrancisco,groupId=2 current=10.3,voltage=219,phase=0.31 1648432611249
In this example, meters is the measurement name, location and groupId are tags, current, voltage, and phase are data fields, and the final value is a millisecond-precision timestamp. TDengine parses these elements and builds the storage structure automatically.
Processing rules for schemaless writes
TDengine applies five rules when handling a schemaless write request:
- Rule 1: Subtable names are auto-generated. The name is an MD5 hash of the measurement name and tag values, prefixed with
t_. - Rule 2: Supertables are auto-created. If the supertable does not exist, TDengine creates it from the first write.
- Rule 3: Subtables are auto-created. If the subtable does not exist, TDengine creates one and links it to the supertable.
- Rule 4: Columns are append-only. New columns are added on demand. Existing columns cannot be removed through the schemaless path.
- Rule 5: Column width auto-extends. When BINARY or NCHAR data exceeds the current column width, TDengine widens the column.
Protocol compatibility
TDengine supports three protocol formats for schemaless writing:
- InfluxDB line protocol: Concise syntax with good readability. Teams already on InfluxDB can switch to TDengine with almost no code changes.
- OpenTSDB Telnet line protocol: Matches OpenTSDB data formats, giving teams a direct migration path.
- OpenTSDB JSON format protocol: Sends data as JSON, which works well for structured data pipelines.
Supporting multiple protocols means TDengine slots into an existing IoT ingestion stack without forcing a rewrite of the data collection layer.
Data consistency
When using schemaless writes, two consistency characteristics matter:
- Idempotency: TDengine guarantees write idempotency. Repeating the same write request does not create duplicate data.
- No multi-row atomicity: Schemaless writes do not guarantee atomicity across multiple rows. Some rows may succeed while others fail. Build error handling and retry logic into your application.
Best practices for IoT
For IoT data ingestion with schemaless writing, these practices help:
- Design tags with care. Tags define how TDengine partitions data. Use device identifiers and location metadata as tags. Keep collected values as data columns.
- Watch column growth. TDengine adds columns automatically, but frequent schema changes can slow writes. Define a complete list of data points before bringing devices online whenever that is practical.
- Use the protocol compatibility. If your system already speaks InfluxDB or OpenTSDB, reuse the existing write logic. Connecting to TDengine takes minimal effort.
- Handle write failures. Multi-row writes are not atomic. Check write results in your application and implement a retry mechanism.
Summary
Schemaless writing is a core time-series database feature for IoT workloads. It simplifies ingestion by automating table creation, column addition, and schema changes as data arrives. TDengine TSDB’s implementation gives teams working with large device fleets a practical path to get data into a time-series database without manual schema management.
