A backfill job can finish successfully and still leave a destination with an incomplete or misleading history. A connector may have transferred every record it received, while the source query returned interpolated values instead of archived events. A timestamp may have lost precision, a quality state may not have a destination equivalent, or a handoff to live ingestion may have left a small gap.
Historian backfill validation is the process of checking that a defined historical dataset reached its destination with the meaning and fidelity required by its users. The goal is not to prove that every possible feature of the source historian has been reproduced. It is to establish what was copied, what was not, and whether the result is fit for a specific use.
The workflow below assumes an existing historian remains available for comparison. If it will be retired, agree on and preserve the evidence needed for reconciliation before access is removed.
Define what a successful copy means
Start with a small, documented scope: source system and version, source tags, destination identifiers, time range, intended use and responsible owners. Record whether the source of record remains the original historian and how long the destination must retain the copy.
Then agree on the retrieval and storage semantics. Is the requirement to copy archived events, sampled or interpolated values, aggregates, or a particular view exposed by an API? A comparison between an archive event stream and a regularly interpolated trend can produce different counts and values even when both queries work as designed. Check source compression, retrieval mode, destination transformations and precision before treating a difference as a defect.
For each signal, capture at least its source identifier, destination mapping, measurement meaning, engineering unit, timestamp interpretation, expected data type and available quality information. If units or values are transformed, document the rule and accepted tolerance. Preserve a link to the original tag even when the destination uses a standardized asset name.
Agree on acceptance criteria before transfer. For example: every critical tag must have the agreed coverage; no unexplained gap over the agreed threshold may remain; selected events must match within an approved precision or conversion tolerance; and quality states required by the use case must either map correctly or be recorded as unsupported. These are examples, not universal thresholds.
Build a reconciliation manifest
Break the scope into manageable time windows and keep a record for each tag or suitable tag group:
| Record | What to capture |
|---|---|
| Scope | Source and destination identifiers, query mode, time zone and window boundaries |
| Transfer | Job ID, configuration version, start and finish time, retry or failure status |
| Source evidence | Available time range, retrieval method, count when comparable, selected test records and known gaps |
| Destination evidence | Comparable query and count, selected records, quality representation and transformation used |
| Decision | Discrepancy, investigation owner, disposition, acceptance and date |
When both interfaces permit it, use consistent half-open windows such as [start, end) so adjacent windows have an unambiguous boundary. Some APIs use inclusive end times or different time-zone rules; adapt the query and record those semantics rather than assuming the notation is supported. Keep the underlying source and destination query parameters with the results so another engineer can repeat the check.
Counts are a useful first signal, not proof of fidelity. A destination may collapse records with the same key, or a source query may include boundary events differently. Conversely, matching counts can hide values attached to the wrong tag or timestamp. Compare equivalent datasets, then investigate mismatches at record level.
Reconcile a representative set before expanding
Consider a six-month transfer of pump discharge pressure from a site historian to a shared analytics store. The historian records irregular archived events; the application needs the recorded history, its source time, unit and quality. The engineer first confirms that the export returns archived events, identifies the target tag and agrees on timestamp precision and quality mapping.
Select windows that include ordinary operation, a known shutdown, a state or quality change, a period of high event density and a known outage if available. Do not rely only on evenly spaced samples: edge cases are often where differences matter.
- Check coverage. Compare the first and last available source records with the destination and identify missing windows. A source with no recorded event in a period is not automatically evidence of a transfer failure.
- Compare equivalent records. Trace selected source tag, timestamp, value, unit and quality to the mapped destination. Apply only the documented transformation and tolerance. Investigate missing, extra or changed records.
- Test precision and ordering. Check events close together, time-zone conversion and daylight-saving transitions where relevant. Do not use arrival order as a substitute for source time. If the target key or precision cannot represent distinct source events, record the limitation and determine whether the intended use can tolerate it.
- Review identity and context. Verify the destination points to the right equipment and measurement, including tags renamed or equipment replaced during the interval. A correct value on the wrong asset is still a failed mapping.
- Resolve exceptions. Classify each difference as a query mismatch, known source gap, mapping or transformation issue, transfer failure, or unsupported behavior. Assign an owner and retest after correction; retain accepted exceptions with their impact.
For a concrete example of why precision deserves attention, the FactoryTalk Historian-to-Historian interface guide warns that truncating subsecond timestamps can cause data loss in its history recovery and archive scanning. That behavior is specific to the documented interface; test the actual systems in use.
Test the boundary between history and ongoing ingestion
A historical transfer and ongoing ingestion are separate operations. Agree on a cutover time, who controls it and how the two paths cover the boundary. If supported, a deliberate overlap can make checking easier, but it also requires a defined way to identify and handle duplicates. A timestamp alone may not uniquely identify an event; use the source and destination’s documented identity and collision rules.
For example, start ongoing collection, backfill through an agreed boundary, then compare a small interval before and after it. Confirm that the destination contains the expected records, that retries do not silently create or overwrite unacceptable results, and that the live path continues after backfill completes. The exact sequence depends on source interfaces and connector behavior; it is not a guarantee that every connector can run both tasks concurrently.
Also test a late-arriving record and, if required and supported, a source correction or deletion. Determine whether the destination receives the change, needs a scheduled re-read, or cannot reproduce it. Historical systems differ in how they expose modifications; the OPC UA Historical Access specification describes one such model, not a universal connector behavior.
Sign off the dataset, not just the job
Before extending the transfer, review the manifest with the source owner and destination owner. Record the tested scope, acceptance results, unresolved gaps, unsupported quality or correction behavior, and the person responsible for ongoing monitoring. Agree who investigates future discrepancies and what happens when source access or retention ends.
A completed job is evidence that a transfer ran. A defensible acceptance decision also shows which records were compared, which exceptions remain and whether the resulting history supports the intended engineering task.


