Curve Fitting: How to Tell If Your Trading Strategy Is Overfit

You cannot know with certainty — overfitting is never directly visible in the backtest that produced it — but you can detect its fingerprints. The three most reliable signals are: performance that collapses when you nudge any parameter slightly (no plateau), a strategy whose number of tuned inputs is large relative to its trade count, and results that degrade sharply on data the optimizer never saw. If your strategy shows any of these, treat the backtest as a description of noise rather than of an edge.

The underlying problem is simple and brutal. Any sufficiently flexible rule set, tuned long enough on a finite price history, will eventually fit the random wiggles of that history instead of any repeatable structure. The equity curve looks beautiful precisely because the parameters were chosen to make it look beautiful. Bailey, Borwein, López de Prado, and Zhu made this formal in "Pseudo-Mathematics and Financial Charlatanism" (Notices of the AMS, 2014): as the number of strategy configurations tried grows, the expected maximum Sharpe ratio found on pure noise grows with it. Search hard enough and an impressive backtest is guaranteed — no market edge required. They also derived a minimum backtest length: the more trials you run, the more history you need before the best in-sample result deserves any belief at all.

So the practical question is never "is my equity curve good?" It is "how many things did I try before this one, and does the result survive tests it was not optimized for?" The rest of this article turns that question into concrete diagnostics a Pine Script or MQL author can run without a research department.

The cheapest diagnostic is parameter sensitivity. Take every tunable input in your strategy — lengths, thresholds, multipliers — and re-run the backtest at neighbouring values: one step up, one step down, then a coarse grid around the chosen point. A strategy that captures something real about market behaviour usually shows a plateau: a broad region of parameter space where results are similar, because the underlying effect does not care whether your lookback is 48, 50, or 53. A curve-fit strategy shows a spike: length 50 is spectacular, 45 and 55 are mediocre or losing, because 50 happens to thread the specific noise of the sample.

The reasoning is straightforward. Markets are noisy, and any genuine regularity is blurry — it cannot reward one exact parameter value and punish its neighbours. Only noise has that kind of resolution. When your optimizer lands on an isolated spike, what it found is a coincidence with an address.

Run the test honestly: vary parameters jointly as well as individually, because interactions hide fragility (the stop looks robust only while the lookback stays at its tuned value). Look at more than net profit — trade count, drawdown, and the shape of the equity curve should also be stable across the neighbourhood. And resist the urge to re-optimize inside the sensitivity scan; the point is to observe fragility, not to fix it by searching harder, which only compounds the multiple-testing problem quantified by Bailey and his co-authors.

If your platform makes this tedious, that tedium is informative: a strategy nobody has stress-tested around its chosen settings should be assumed fragile until shown otherwise.

Every input you tuned, every filter you added after seeing results, every asset or date range you selected because it looked better — each is a degree of freedom, a knob the optimization process could turn to flatter the sample. The data pushing back against those knobs is your set of independent trades. When degrees of freedom are large relative to trade count, the fit is nearly unconstrained: the optimizer has enough freedom to explain the sample without needing a real effect.

A retail script with an entry length, a confirmation filter, an ADX threshold, a session filter, a stop multiple, a target multiple, and a "skip Mondays" rule has seven-plus effective degrees of freedom. If it produced 60 trades in the backtest, that is fewer than nine trades of evidence per fitted choice — and the trades are usually not independent, since overlapping positions and regime clustering shrink the effective sample further. There is no magic ratio, and reputable researchers avoid pretending there is one, but the direction is unambiguous: more knobs demand disproportionately more trades, and Bailey et al.'s minimum backtest length result formalizes exactly this — required evidence grows with the size of the search.

Two honest disciplines follow. First, count your hidden trials: every discarded variant you tried before the current one is part of the search, even if it no longer appears in the code. Second, prefer fewer inputs: a two-parameter strategy with 400 trades earns more belief than a nine-parameter strategy with 80, regardless of which equity curve looks prettier. Deleting a filter that only helps in-sample is one of the few free improvements in strategy research.

The classical defence is out-of-sample (OOS) testing: fit on one segment of history, evaluate on a held-out segment the optimizer never touched. Walk-forward analysis repeats this rolling forward — optimize on a window, trade the next window, slide, and judge only the stitched-together out-of-sample results. Sharp degradation from in-sample to OOS is the signature of curve fitting. But there is a catch every practitioner eventually meets: if you peek at OOS results and go back to tweak, the hold-out silently becomes part of the training data. One honest look is all a hold-out set gives you.

Academic finance has spent decades building tools for the harder problem of data snooping — searching many strategies and reporting the best. Halbert White's "Reality Check" (Econometrica, 2000) tests whether the best rule from a searched universe genuinely beats a benchmark once the full breadth of the search is accounted for; Peter Hansen's SPA test (2005) sharpened it against being diluted by obviously poor rules. Bailey and López de Prado's Deflated Sharpe Ratio (2014) adjusts a reported Sharpe for the number of trials, their variance, and the non-normality of returns — asking whether the best result exceeds what mining noise would produce. Their related Probability of Backtest Overfitting framework (with Borwein and Zhu, using combinatorially symmetric cross-validation) estimates how often the in-sample winner underperforms out of sample.

You do not need to implement these papers to use their lesson: a backtest metric is uninterpretable without knowing how many attempts stand behind it, and honest evaluation charges you for every attempt.

Condensed into a checklist you can run on any TradingView or MetaTrader strategy:

1. Count your knobs. List every input plus every after-the-fact filter and date/asset choice. If the list is long relative to your trade count, assume overfitting by default.
2. Run the plateau test. Perturb each parameter, then pairs of parameters. Spikes are disqualifying; plateaus are merely encouraging.
3. Hold out data and spend it once. Reserve a final segment (and ideally other symbols) untouched during development. Evaluate there a single time, and treat any post-peek tweaking as starting over.
4. Walk it forward. Judge the concatenated out-of-sample windows, not the optimization windows.
5. Account for trials. Keep a log of every variant tested. In the spirit of the Deflated Sharpe Ratio, discount the winner by the depth of the search that produced it.
6. Verify the code before the statistics. Repainting inputs, lookahead in request.security(), and unwired stop inputs all fabricate performance that no amount of statistical care can rescue. ForexCodes' Strategy Validator exists precisely for this layer — checking that the backtest measures what the code claims — because correctness is verifiable even though profitability is not.

And the framing to keep: passing every test above does not prove an edge. It only means the strategy has not yet been rejected. This article is educational only, not financial advice; markets change, a clean backtest is not a forecast, and trading always carries a real risk of loss.