To keep a downstream copy current, discover changes to previously copied records and apply them using a stable record identity and an explicit revision policy. Advancing a query only by measurement time will miss a correction made today to last month’s value. The transfer also needs a defined treatment for deletions and any calculations that used the earlier value.
For a historian engineer, this is a continuing synchronization task after the initial backfill. The source may be a historian, a lab system or another authorized operational record store. The available interface determines how much of the procedure can be automated.
Keep event time separate from change position
An observation can have several useful times: when the sample or event belongs in the process history, when it was changed, and when the downstream system received it. These fields answer different questions.
The integration needs a reliable way to find changes since its last completed synchronization. A source may offer an ordered change sequence, a versioned export, a modification timestamp with documented guarantees, or no suitable change feed at all. Do not substitute the greatest measurement timestamp for that position.
The public LIMS-to-PI correction discussion describes a concrete failure: an edited lab value did not update its PI copy. Suggested rereads and database approaches demonstrate possible designs, not a verified universal fix for OPC HDA interfaces.
Choose a mechanism the source actually supports
| Mechanism | What it can provide | Boundary to establish |
|---|---|---|
| Ordered change feed or database change tracking | Identifies changed keys and potentially deletion operations | Retention, checkpoint validity, ordering and current-value retrieval |
| Modification-time query | Finds rows changed after a recorded position | Timestamp ties, commit visibility, clock behavior and deleted rows |
| Re-read and compare historical windows | Detects differences in the reread scope | Consistent source view, permitted query load and maximum correction age |
| Manual revision notification | Supplies approved correction scope when no machine-readable mechanism exists | Named owner, response time and traceable re-extraction |
For example, SQL Server Change Tracking uses synchronization versions and requires checking whether an older version is still valid. It can identify deleted keys; current rows and consistent enumeration require additional handling. This is an example for an authorized, configured SQL source, not a feature to assume in a historian connector or to enable in a vendor database without approval.
A modification timestamp is weaker than a documented ordered feed. Two edits may share a timestamp; a transaction can commit after an enumeration has passed its apparent time. Use the source’s consistency protocol. An overlap and deduplication strategy can reduce risk, but it does not prove completeness without a bound on delayed visibility. Hard deletions need a retained deletion marker, change log or authoritative comparison.
Work through a correction stream
This fictional source provides immutable observation IDs, monotonically ordered revision numbers and retained deletion operations. The destination keeps both received revisions and a current projection.
| Revision | Observation | Event time | Operation | Payload |
|---|---|---|---|---|
| 101 | O17 | Monday 10:00 | Insert | 8.0 mg/L, Good |
| 102 | O18 | Monday 10:05 | Insert | 7.0 mg/L, Good |
| 103 | O17 | Monday 10:00 | Replace | 8.4 mg/L, Good |
| 104 | O18 | Monday 10:05 | Delete | Observation withdrawn |
After revision 102, the current projection contains O17 = 8.0 and O18 = 7.0. After 104, it contains O17 = 8.4 and an explicit withdrawn state for O18. A consumer must not interpret the withdrawn reading as a measured zero.
If revision 103 is replayed, the same observation and revision should not create another current reading. If 102 arrives after 104, the older revision must not resurrect O18. Those outcomes depend on the example’s source revision ordering; they cannot be inferred from arrival order.
Persist the applied revision and checkpoint so a restart can replay safely. Where source and destination cannot share one transaction, retain a durable intake log and use replay-safe application before advancing the intake checkpoint. Keep errors visible rather than silently skipping a revision and reporting the stream current. For a TDengine TSDB destination, check its primary-key and partial-write rules.
A modified-history read is not automatically a change feed
OPC UA Historical Access provides Read Modified functionality for supported historical nodes. Its historical-time domain and modification records need careful interpretation: a replacement record can contain the old value that was changed, not the current replacement. Multiple modifications and retained detail are server-dependent.
Do not treat that response as a globally ordered “changes since last synchronization” stream. Establish how to enumerate affected historical times, obtain the current value or deletion state, and distinguish the operations the server exposes. The specification’s update-type enumeration distinguishes insert, replace, update and remove; it does not establish that every server or connector supports them.
If the only permitted strategy is to reread the previous seven days, document the consequence: corrections older than seven days are outside automatic discovery. Add periodic wider reconciliation or a manual process if those changes matter. Do not describe a lookback as complete unless the business and source constrain correction age accordingly.
Carry the revision into derived results
List the reports, aggregates and extracts that depend on the corrected observation. In the example, an arithmetic average of the two initially valid readings would change from 7.5 mg/L to 8.4 mg/L after replacement and withdrawal, assuming the report excludes withdrawn values. Its accepted sample count changes from two to one. The revised report should expose that change rather than merely replacing the displayed average.
Recompute the affected scope under the same documented quality and calculation policy, retain lineage to the input revisions, and mark an earlier delivered result superseded when appropriate. Retract or flag a result if a required input has been withdrawn and no valid calculation remains. Record gaps in change discovery separately from normal transfer delay.
Test replacement at an old event time, replay, out-of-order revision delivery, deletion, a correction beyond the lookback, and an expired source checkpoint. Acceptance means the downstream current state and dependent results reflect the supported revision contract. The multi-site integration article provides broader ownership context; the correction contract should name who investigates every unresolved divergence.


