Before You Buy a Trading Strategy: A Backtest Verification Checklist

You verify a strategy by reconciling its claimed numbers against its own trade list, checking whether the sample is large enough to support any claim, and demanding evidence that the results hold on data the strategy never saw. If a seller cannot give you a full trade-by-trade history and an out-of-sample result, you have not been given enough to verify anything, and the honest conclusion is to walk away rather than to trust the headline.

Most strategy, EA, and signal pitches lead with a big net-profit figure and a high win rate over a hand-picked window. Those are the two numbers easiest to make look good and least informative about robustness. The checklist below is built to get underneath them. It will not tell you whether a strategy will make money — nothing can, and anyone who promises that is selling you the promise, not the strategy. What it can do is separate results that were honestly measured from results that were curve-fit, cherry-picked, or corrupted by look-ahead bias. That distinction is the whole job. This is a due-diligence guide for educational purposes and is not financial advice. Run the seller's numbers through the free [Backtest Health Check](/backtest) as you work down the list, and read [why most backtests fail](/learn/why-most-backtests-fail) first for the failure modes each check targets.

Ask for the full trade-by-trade export, not a summary screenshot. Then reconcile. The seller's headline metrics have to be reproducible from the raw trades, and if they are not, one of the two is wrong.

The arithmetic is simple and worth doing by hand at least once:

• Sum of all winning trades divided by absolute sum of all losing trades should equal the stated profit factor.
• Total net across all trades divided by trade count should equal the stated expectancy.
• Peak-to-trough on the running equity should equal the stated max drawdown.

When these do not tie out, the summary was computed on a different data set, a different period, or with costs quietly removed. Speaking of costs: check that spread, commission, and slippage were modeled. A backtest run at zero cost can turn a losing system into a winning-looking one, and on a high-frequency EA the costs often dwarf the edge. Confirm the fills are realistic too — market orders that always fill at the exact signal price are a red flag. Reconciliation is unglamorous and it catches more bad strategies than any clever statistic, because a seller who fudged the headline usually did not bother to also fudge a consistent trade list.

Count the trades before you read any metric. A backtest with 40 trades cannot support a confident claim about anything; the uncertainty on every figure is too wide, and a handful of lucky trades can carry the entire result. Uncertainty shrinks only with the square root of trade count, so ten times the confidence needs a hundred times the trades. For where the thresholds sit, see [how many trades is enough](/learn/how-many-trades-is-enough).

Then run the drop-the-best-K test. Remove the single most profitable trade and re-read expectancy and profit factor. Then remove the top three. Then the top five. If a strategy's edge collapses when you delete its best few trades, the edge was those trades, not the system — you would be buying a lottery result, not a repeatable process.

| Reading | Net profit | Profit factor | |---|---|---| | All trades | 100% | 1.8 | | Drop best 1 | 62% | 1.4 | | Drop best 5 | 5% | 1.03 |

A table like this is a warning, not a verdict, but it reframes the pitch honestly: most of the headline came from five trades out of however many. Ask whether you believe those five repeat. Often the honest answer is that you have no way to know, which is itself a reason for caution.

Look-ahead bias is when a backtest uses information that would not have been available at decision time. It is the most common reason a strategy that tested beautifully falls apart live, and it is often invisible in the summary. The tells you can check for:

• Signals that reference the close of the bar they trade on. If an entry fires on a candle's close but the backtest assumes you were already in during that candle, the result used the future.
• Indicators that repaint. A repainting indicator changes its past values as new bars arrive, so the historical chart shows signals that were never actually present in real time. Non-repainting is a claim to verify, not accept.
• Whole-series normalization or scaling. If features were normalized using statistics computed over the entire history, every early bar quietly knows the future distribution.
• Suspiciously smooth equity curves and win rates above roughly 90%. These are not proof of look-ahead, but they are the signature it leaves, and they warrant hard questions.

For a full treatment of how this creeps in and how to test for it, see [look-ahead bias](/learn/look-ahead-bias). If the strategy is code you can run, the cleanest test is to feed it bar-by-bar in strict time order and confirm no decision ever touches a future bar. If the seller cannot or will not let you run it that way, treat the results as unverified.

In-sample results are the ones measured on the same data used to build and tune the strategy. They are almost always flattering, because a strategy tuned hard enough will fit the noise in its training window and report a wonderful backtest that carries no real edge. This is the overfitting problem, documented in depth by Bailey, Lopez de Prado, David Aronson, and Campbell Harvey — Harvey and Yan Liu in particular have shown how testing many strategies and reporting only the winner manufactures significance that vanishes out of sample. The defense is to insist on results from data the strategy never touched during development.

Concretely, demand one or more of: a genuine out-of-sample window held back from tuning, a walk-forward analysis that re-optimizes on a rolling basis and reports only the forward segments, and honest treatment of how many configurations were tried before this one was chosen. A single walk-forward chart tells you more than any in-sample headline. See [in-sample vs out-of-sample](/learn/in-sample-vs-out-of-sample) and [walk-forward analysis](/learn/walk-forward-analysis) for what good looks like, and run the free [Overfitting Check](/backtest/overfitting) on any result the seller provides.

One closing point, because it is the point: passing every check on this list means the strategy was honestly measured and is not obviously broken. It does not mean the strategy is profitable, and it never will. Verification is about integrity and robustness, never a promised return. This checklist is educational and is not financial advice; the most valuable outcome it produces is often the confidence to say no.