Historical testing
Test deterministic L3 behavior and fake-provider responses without real model calls during replay.
Historical tests can exercise L3 hooks and policies with deterministic inputs. Real provider calls are disabled during backtesting and historical warmup, including when source configuration requests a real provider.
Start with request boundaries
Use the deterministic request guard. Exercise allowed and rejected quantities with the same SJS signal inputs, then test explicit close and delegated placement behavior.
Check missing quantities, numeric strings, zero, negative values, and non-finite values. Test the absence of a prior frame hook. Observe actual orders and positions rather than only the hook's Boolean.
Use a fake provider
The supported fake adapter can supply output, simulate a delay, or throw a specified error. The example records one validated provider observation per session without requesting execution.
Fake output exercises your response handling. It does not establish the quality, latency, tool behavior, or reproducibility of a real provider. If you supply tools and requested tool calls to a fake request, those tools can execute; “fake provider” is not a universal no-side-effects switch.
Separate kinds of validation
Documentation fixture tests check the example's decisions and state. A historical engine run checks how it interacts with strategy signals, runtime controls, and simulation. A supported live provider configuration is a separate integration concern.
Agent Lab's assistant can edit L3 source and run historical experiments, but its authoring provider does not override the restriction on real provider calls made inside the strategy.
Record enough to compare
Save the L3 and SJS revision, configuration, input data selection, execution assumptions, request kind/source, hook result, and final execution outcome. Fresh runs start fresh runtime memory unless a separate supported mechanism says otherwise.
Continue with examples.