Market Replay vs Strategy Analyzer: Which NinjaTrader Backtest to Trust

The Strategy Analyzer runs a bar-based backtest: it feeds your strategy historical bars and, by default, evaluates and fills using open/high/low/close values rather than the underlying tick stream. Market Replay instead plays back recorded tick-by-tick market data in sequence, so your strategy processes the same granular price movement it would see live, one tick at a time. In short, the Strategy Analyzer is fast and coarse; Market Replay is slow and faithful.

The consequence is what each can observe. Under bar-based backtesting, everything that happens inside a bar is invisible unless you explicitly raise the resolution — the strategy sees four price points per bar. Market Replay reconstructs the intrabar path, including the order in which the high and low were reached, which is exactly the information a stop or a Calculate.OnEachTick strategy depends on.

Neither tool is inherently "the truth," but they answer different questions. The Strategy Analyzer answers "how does my logic behave on closed bars, quickly, across a long history?" Market Replay answers "how would this strategy have actually processed the live tick stream?" When those answers disagree, that disagreement is diagnostic.

The Strategy Analyzer excels at breadth. It can sweep years of data in seconds, run parameter optimizations across thousands of combinations, and give you a fast read on whether logic that operates on completed bars is even wired up correctly. For a strategy that trades strictly on Calculate.OnBarClose and references only confirmed bars, the Strategy Analyzer's results can be quite representative.

What it hides is intrabar reality. By default it does not know whether the high or the low came first within a bar, so its fill assumptions for stops and targets can be optimistic — it may assume a favorable ordering that never occurred. It also cannot expose intrabar repainting: if your strategy makes decisions on a forming bar under OnEachTick, the Strategy Analyzer's coarse feed can mask signals that would flicker live.

You can improve fidelity with the "tick replay" backtest option or by adding a fine-grained secondary series, which forces intrabar processing. But absent that, treat Strategy Analyzer output as a screen, not a verdict. Impressive numbers there are a reason to investigate further, not to trust — a theme covered in depth in [why most backtests fail](/learn/why-most-backtests-fail).

Market Replay's value is that it reconstructs the intrabar sequence. Because it plays real recorded ticks, it shows you the actual path price took to reach the bar's high and low, in order. That path is what determines whether a stop was hit before a target, whether a limit order would have filled, and whether a signal computed on a live bar was stable or fleeting.

This makes Market Replay the honest test for two kinds of strategy: those that fill intrabar (any stop, target, or stop-market exit), and those that compute on every tick. If your strategy uses Calculate.OnEachTick and reads Close[0] while the bar is still forming, only Market Replay will surface the fact that the value was moving. The Strategy Analyzer, seeing one settled close per bar, cannot reproduce that instability.

Market Replay is slower and requires downloaded replay data for the instrument and dates you want, so it is not a tool for sweeping years or optimizing parameters. Its role is confirmation: once the Strategy Analyzer says a strategy is worth a closer look, Market Replay tells you whether that behavior holds up when price arrives one tick at a time.

Here is the core diagnostic. If a strategy references only confirmed data — bar 1 and older, decided on bar close — its entries and exits should be essentially identical in the Strategy Analyzer and in Market Replay. The two runs may differ slightly in fill price due to intrabar path, but the set of trades and the signals that generated them should match.

When they diverge sharply — trades that appear in one run and not the other, or signals that the Strategy Analyzer shows but Market Replay does not — the usual cause is [repainting](/learn/repainting). The strategy made a decision on a forming bar 0 that looked true on the coarse feed but flickered away under real ticks, or it relied on information that would not have existed at decision time, which is [look-ahead bias](/learn/look-ahead-bias). Consider the classic offender:

// WRONG: acts on the live, unconfirmed bar under OnEachTick
protected override void OnBarUpdate()
{
    // Close[0] is still forming; this can be true mid-bar and false at close
    if (Close[0] > High[1])
        EnterLong();
}

In the Strategy Analyzer this may fire cleanly on a favorable close; in Market Replay it fires, reverses, or vanishes as the tick stream moves. That gap between the two tools is the fingerprint of repainting.

Treat them as a pipeline, not alternatives. Use the Strategy Analyzer first for breadth: verify the logic runs, screen across a long history, and — carefully — narrow parameters. Then use Market Replay as the confirmation gate: pick representative dates, replay the ticks, and check that the trades and signals reproduce what the bar-based test claimed. A strategy that survives both, referencing only confirmed bars, is one whose test at least describes behavior that could have occurred live.

To write logic that matches across both tools, decide on closed data. The confirmed version of the buggy example above gates on the first tick of a new bar and reads bar 1:

protected override void OnBarUpdate()
{
    if (CurrentBar < 2)
        return;

    if (IsFirstTickOfBar && Close[1] > High[2])
        EnterLong();
}

This fires on settled values, so the Strategy Analyzer and Market Replay agree. Beware of one honest trap: optimizing heavily in the fast bar-based tool can curve-fit parameters that only Market Replay reveals as fragile — breadth without confirmation invites overfitting, another face of [why most backtests fail](/learn/why-most-backtests-fail).

Running both tools and comparing is the empirical way to catch repaint, but it is slow and requires replay data. Many of the underlying causes are visible statically, before you download a single tick: decisions on unconfirmed Close[0] under OnEachTick, negative or future indexing, exit levels computed from forming bars, and secondary-series misalignment. A code scan can flag these patterns and tell you in advance that a Strategy Analyzer / Market Replay discrepancy is likely.

That is what the ForexCodes analyzer does. Paste a strategy into the [free NinjaScript validator](/ninjascript) and it flags intrabar-repaint and look-ahead patterns without you having to set up and reconcile two backtests by hand. It is a first line of defense, not a replacement for Market Replay confirmation — but it turns a slow empirical hunt into a fast static check.

Finally, keep the goal in view. Reconciling these tools does not make a strategy profitable; it only ensures your test describes a strategy that could actually have traded, with honest intrabar fills and no repainting. Agreement between the Strategy Analyzer and Market Replay is a credibility test, not a performance promise. Nothing here is trading advice, and no result implies any win rate or return.