An MQTT broker moves application messages between publishers and subscribers. A historian preserves time-indexed records so engineers can query, trend, aggregate, and investigate them later. These responsibilities are complementary, but they are not interchangeable.
A broker can deliver messages reliably and still leave gaps, duplicates, or unusable rows in the historian. Conversely, a historian can retain years of data even though it did not participate in the original device-to-broker delivery. The architecture becomes easier to validate when each component has a clear job.
Broker and historian responsibilities
| Responsibility | MQTT broker | Historian or time-series database |
|---|---|---|
| Connect producers and consumers | Primary role | Not normally its protocol role |
| Route data | Matches published topics to subscriptions | Organizes committed records for storage and access |
| Handle MQTT delivery state | QoS acknowledgements, sessions, subscriptions, and broker queues | Outside MQTT protocol handling unless implemented by an ingestion component |
| Preserve long-term time series | Not the core MQTT function | Primary role |
| Apply retention by engineering policy | Message and session expiry are transport controls | Data retention, tiering, and deletion are storage policies |
| Support historical analysis | Limited to broker-specific operational features | Time-based queries, filtering, aggregation, downsampling, and analysis |
The distinction is architectural, not a judgment that one component is more important. The broker decouples systems in motion. The historian makes selected data durable and useful after the message exchange is over.
Why broker retention is not historical storage
MQTT includes several storage-related mechanisms, but none turns the protocol into a historian.
A retained message is the broker’s current retained message for a topic. A new matching subscriber can receive it immediately instead of waiting for the next publication. Replacing that retained message normally replaces the prior retained state; it does not create a queryable series of every value.
A persistent session can preserve subscriptions and eligible queued or unacknowledged messages while a client is offline. Its purpose is to continue a messaging session. Session expiry, broker capacity, message expiry, and administrative policy can all limit what remains available.
Broker persistence protects broker state across restart according to the broker’s implementation and configuration. It can be essential for reliable messaging, but it still does not define table schemas, engineering timestamps, years-long retention, SQL queries, aggregates, or audit-ready change history.
What happens after the broker delivers a message
Once a subscriber receives a message, several steps remain before usable history exists:
- The subscriber must accept the correct topic and payload version.
- Parsing must extract equipment identity, measurement values, units, quality, and timestamps.
- Mapping must send those fields to the intended database, table, columns, and tags.
- The write path must handle timeouts, retries, duplicate messages, and rejected records.
- Storage must commit the record and retain it under an explicit data policy.
An acknowledgement at the MQTT layer does not verify the last four steps. Even QoS 2’s exactly-once behavior applies to one MQTT sender-receiver delivery flow, not to the subscriber’s database transaction.
This is also why a “connected” dashboard is weak evidence. It proves that some part of the live path works. It does not prove that every source message became one correct, durable row.
Historical integrity depends on data decisions
The ingestion layer creates the boundary between message delivery and history. Five decisions have an outsized effect.
Schema mapping
Plain MQTT does not define an industrial payload schema. A topic and payload may identify a site, compressor, pressure value, unit, and operating state, or leave some of those meanings implicit. The ingestion process must validate the payload version and map each field consistently.
Timestamp selection
The time the broker receives a message, the time the collector processes it, and the time the device measured the value can differ. During an outage, replayed values may arrive minutes or hours late. A historian needs a rule for event time, ingestion time, clock quality, time zone, and invalid timestamps.
Duplicate handling
QoS 1 permits duplicate delivery. Application retries and database retries can also repeat data independently of MQTT. A source sequence number, stable event ID, or appropriate database key can help distinguish a retransmission from a legitimate repeated value.
Retention and recovery
Broker queue limits protect a messaging service; historian retention rules protect long-term records. The system must define how much data can be buffered at the source, broker, and collector, how replay is ordered, and what evidence confirms recovery.
Query behavior
Historical usefulness depends on more than storage. Engineers need to filter by asset and time, aggregate intervals, compare operating periods, and detect gaps. Those are database and data-model responsibilities, not MQTT delivery semantics.
A concise industrial architecture example
Consider three compressed-air stations publishing pressure, flow, power, and compressor state to a central broker. Maintenance dashboards and alerting services subscribe for current operations. A separate ingestion service subscribes to the same approved topics, validates the JSON schema, extracts and validates source timestamps when available, and writes the measurements to a historian.
If a dashboard is offline, a persistent session may help it receive eligible queued messages later. If the historian ingestion service is offline, broker queues may reduce loss within configured limits. If a station loses its upstream connection, neither mechanism captures measurements unless the station or gateway buffers them locally.
After reconnection, the ingestion service must still replay in a controlled way, reject invalid payloads visibly, and prevent retries from corrupting historical totals. The broker and historian solve different parts of the same pipeline.
When both components belong in the design
Use a broker when several producers and consumers need decoupled, topic-based message exchange. Use a historian when operations require durable records, time-based queries, governed retention, and analysis across equipment and sites. Most industrial MQTT architectures need both, plus an ingestion layer with explicit transformation and recovery rules.
The separate decision between source protocols is covered in OPC UA vs. MQTT for industrial data collection. Once MQTT is selected, the broker-versus-historian boundary still has to be designed.
TDengine documents an Enterprise MQTT ingestion connector that subscribes to broker topics, parses topic and payload fields, maps them to tables, and writes them into TDengine. This is an inbound collection path; TDengine’s outbound MQTT publishing capability is separate. In an architecture using TDengine TSDB for time-series storage and queries, the broker remains the message-routing component while the connector and database create durable history.
Validate the complete pipeline
Before deployment, verify the system with evidence from source to query result:
- Record the owner and retention limit of every source, broker, and collector buffer.
- Compare published message counts, subscriber receipts, accepted writes, rejected writes, and stored rows.
- Interrupt each connection and component, then verify replay order, duplicates, gaps, and recovery time.
- Query representative periods using the source timestamp and confirm late data appears where engineers expect it.
- Alert separately on broker health, ingestion lag, mapping failures, database write failures, and historical gaps.
The pipeline is complete only when a source event can be traced to a correct stored record and retrieved through the intended historical query.
