How to Detect Overfitting in an MT5 or TradingView Optimization

If you ran a large parameter sweep and kept the top result, it is likely that a meaningful part of its apparent edge is noise, and the more combinations you tried, the more likely that is. This is not a character flaw in your process; it is arithmetic. When you test hundreds of parameter sets on the same history, some will look excellent purely by chance, and the optimizer's job is literally to hand you the most extreme of them. MT5's Strategy Tester and TradingView's optimization workflows make it trivial to try thousands of combinations, which quietly makes the selection problem worse, not better.

The uncomfortable framing is this: a genetic optimizer over a wide parameter grid is, from a statistics point of view, a machine for maximising in-sample luck. That does not mean every optimized strategy is worthless. It means the raw result the tester shows you is an upper bound distorted by selection, and you cannot know how distorted without measuring it. The number of trials changes the meaning of the winner. Two honest tools measure that distortion, and the rest of this article walks through both. For the underlying intuition on why more testing makes the best result less trustworthy, start with [overfitting and curve-fitting explained](/learn/overfitting-curve-fitting-explained) and [why most backtests fail](/learn/why-most-backtests-fail).

A multiple-testing haircut is a downward adjustment to your headline statistic (usually the Sharpe ratio) that accounts for how many configurations you tried before landing on this one. The logic, developed rigorously by Harvey and Liu and by Bailey and Lopez de Prado, is that a Sharpe of, say, a given size is far less impressive if it is the best of a thousand tries than if it is the only test you ran. The haircut shrinks the number toward what you would expect from the search itself.

The Deflated Sharpe Ratio is the cleanest single expression of this. It adjusts the observed Sharpe for the number of independent trials, the length of the track record, and the skew and kurtosis of the returns, then asks whether what remains is still distinguishable from zero. A strategy whose Sharpe survives deflation has cleared a real bar; one whose deflated Sharpe collapses toward zero was mostly selection luck. Our [deflated Sharpe ratio](/learn/deflated-sharpe-ratio) article walks the mechanics.

The operational catch in MT5 and TradingView is honest trial counting. The relevant number is not the count of variants you finally compared; it is every configuration you tried across the whole search, including the ones you discarded and the earlier sweeps you have half-forgotten. Undercount your trials and the haircut is too gentle, which flatters you again. When in doubt, count generously. The haircut works on aggregate statistics, which is its convenience and its limit: it discounts the summary number without looking at how the variants relate to each other over time. For that you need the second tool.

The second tool operates on the full returns matrix rather than a single summary, and that is what makes it powerful. Instead of one Sharpe number, you keep the per-period return series of every variant the optimizer tested, arranged as a matrix with one column per variant and one row per period. That matrix is the true record of your search.

On that matrix you can run two families of test. PBO via CSCV, from the previous companion article [Probability of Backtest Overfitting](/learn/probability-of-backtest-overfitting), measures how often your in-sample winner drops below median out-of-sample across all symmetric splits. White's Reality Check, and Hansen's Superior Predictive Ability test which refines it, ask a formal question: is the single best variant genuinely better than a benchmark once you account for the fact that you searched the whole matrix to find it? Both use the joint behaviour of all variants, which is exactly the information a per-statistic haircut cannot see.

The practical hurdle is identical to the PBO case: you must retain the returns of every tested variant, not just the winner. MT5 and TradingView will happily show you the champion and let the rest evaporate. Configure your workflow to export or log each variant's equity or return series as it runs. Without the matrix, neither PBO nor Reality Check has anything to work on, and you are left with only the aggregate haircut. Keeping the matrix is the single most valuable habit change for anyone who optimizes seriously.

The two tools cover different blind spots, so run both, and the sequence below is a workable order.

| Step | What you do | What it protects against | |---|---|---| | 1 | Log every variant's per-period returns during the sweep | Losing the matrix the real tests need | | 2 | Count total trials honestly, including discarded sweeps | Under-haircutting the headline Sharpe | | 3 | Apply the Deflated Sharpe / multiple-testing haircut to the aggregate | Being fooled by a lucky summary statistic | | 4 | Run PBO (CSCV) on the returns matrix | Selection that fits noise across splits | | 5 | Run White's Reality Check or Hansen's SPA on the matrix | The best variant beating a benchmark only by luck of search | | 6 | Confirm the inputs are clean | Contaminated returns making every test meaningless |

Step 6 is not optional busywork. All of these tests assume the returns you feed them are real. If your MT5 backtest used the wrong tick model, or your TradingView script referenced a bar that had not closed, the returns are fiction and so is every diagnostic computed on them. Check for [look-ahead bias](/learn/look-ahead-bias) before you trust any number downstream. A common Pine mistake is reading a value that only exists once the bar completes and acting on it within the same bar; if your logic does that, fix it before you optimize, not after.

Walk-forward, from [walk-forward analysis](/learn/walk-forward-analysis), is the procedural complement: it forces out-of-sample discipline across time rather than measuring overfit after the fact. Ideally you do both.

Passing both checks licenses a narrow, honest claim: your best variant's edge is not obviously an artifact of having searched a large space, and its selection carries information across splits. That is worth having, and most optimized strategies never clear it. But be precise about the boundary. A clean deflated Sharpe and a low PBO say the result is robust to the particular ways you can measure overfitting on your own data. They say nothing about whether the market regime that produced your history will continue, whether your costs and slippage were modelled correctly, or whether the strategy will make money. Robustness on historical data is not a forecast.

This is the whole ForexCodes stance, and it is worth stating flatly: these are integrity and robustness tools, never profitability tools. Their entire value is in catching the backtest that lies to you, so you allocate attention and capital to strategies whose reported edge survives honest scrutiny, and you discard the ones that don't. That is a filter against self-deception, not a signal to trade.

If you would rather not wire up CSCV, Reality Check, and the deflation maths yourself, the free [Overfitting Check at /backtest/overfitting](/backtest/overfitting) takes a returns matrix from an MT5 or TradingView export and reports PBO with the companion diagnostics, and the [Backtest Health Check](/backtest) runs the wider integrity scan including look-ahead and sample-size warnings. Both are built to answer one question honestly: how likely is it that this result is fooling you? Everything here is educational and is not financial advice.