Debugging backtests
Locate missing data, skipped frames, warmup issues, blocked requests, and misleading result comparisons.
Reduce the run to a short window containing the first unexpected event. Keep the instrument, source revision, and execution settings fixed while tracing that event through the pipeline.
Locate the first missing stage
| Symptom | Inspect next |
|---|---|
| No data found | Full instrument identity, contract dates, provider coverage, and selected range |
| Data limit reached | Shorten the Lab window or choose an appropriate coarser interval |
| Invalid historical data | Date parsing and finite OHLCV values; do not patch missing values into plausible signals |
| Bars visible but no SJS decisions | Session filters, cadence, source compilation, and input validation |
| Warmup continually restarts | Input interval and actual gaps versus the strategy's gap threshold |
| Signal exists but no request | Position state, pending orders, and the strategy's consumed-signal rules |
| Request not accepted | Automation/L3 gates, duplicate-order checks, and execution availability |
| Accepted request remains pending | Fill delay, quote availability, price thresholds, capital, and trade limits |
| Limit or stop touched by high/low but did not fill | Shared simulation tests the available quote close |
| Last position did not close | Run-end close rejection or delay with no later fill opportunity |
| Logs unavailable | The run did not complete with a saved log file, or its log-retention period expired |
| Same dates, different results | Source, actual data, warmup, runtime, quantity, delay, and session settings |
Use an observation-only run
Temporarily replace SJS with the data observer. It reports the frames that reach the script, including non-increasing timestamps and gaps. It cannot tell you whether a skipped frame was absent in the raw dataset or filtered earlier by the runtime.
Restore the original source in a fresh run afterward. The observer run is a diagnostic, not a comparable trading result.
Explain a pending order
Compare the order's timestamp with simulated market time, then verify a qualifying quote exists. A 90-second delay on one-minute data can first become eligible at the two-minute observation; it does not schedule an intermediate event.
If the order predates a session boundary, inspect the replay loop's skip/close behavior. Do not assume every loaded observation caused a fill check or a strategy invocation.
Explain a blocked close
Inspect both the initial close check and the delegated placement where the shared simulator uses that path. A quantity gate can permit close_position and then reject the resulting place_order. See SJS with L3.
Treat failures as failures
Agent Lab surfaces source compilation and strategy execution failures rather than treating them as successful zero-trade runs. A timeout, cancellation, invalid dataset, or oversized result is a different outcome from a completed run whose strategy chose not to trade.
Real in-strategy AI calls are disabled during replay. An authoring model being connected in Agent Lab does not change that provider restriction.
Continue with SJS debugging for script-level checks or Policy debugging for L3 decisions.