Missing and stale data
Distinguish absent rows, filtered frames, repeated updates, and outdated derived evidence.
Treat data quality as several separate checks. Collapsing every unexpected observation into “stale” makes it harder to identify the right fix.
Classify what happened
| Observation | Possible interpretation | Check next |
|---|---|---|
| No rows in the requested window | Coverage, selection, or request problem | Instrument, bounds, and adapter error |
| Rows exist but no SJS frames | Processing rules filtered them | Session gates and strategy cadence |
| Same timestamp arrives again | Another update for that observation | Producer contract and sampling policy |
| Timestamp moves backwards | Older or out-of-order observation | Data ordering and reconnect behavior |
| Timestamp jumps forward | Session boundary, omitted frames, or missing data | Raw coverage and runtime filtering |
| Derived output has an old timestamp | Producer has not supplied current evidence | Readiness and execution order |
Choose a gap policy
A strategy can reject a decision, restart local warmup, or continue with an explicitly accepted gap. Choose the rule based on what the calculation requires, and keep it with the source revision.
The SJS crossover resets after a gap longer than two minutes. That is suitable only for the sample's stated assumptions. Applying it unchanged to fifteen-minute observations would repeatedly restart the window.
Do not fabricate bars or forward-fill prices silently. If a particular workflow uses interpolation, its outputs need a documented interpretation; Agent Lab's current bar-based workflow rejects interpolation configurations.
Check freshness against the right time
During replay, compare evidence with the decision's observation time. During live processing, distinguish the market timestamp from delivery time. Use the producer's timestamp convention before declaring evidence too old.
Future evidence is also invalid for a historical decision. Confirm that direct historical requests and derived points do not introduce timestamps beyond the decision boundary.
Preserve useful diagnostics
Record the decision timestamp, evidence timestamp, instrument, and reason for rejecting a sample. Keep logs bounded or focused on a chosen window. Avoid logging credentials or dumping entire context objects.
Continue with debugging to trace these checks in order.