The more strategy variants you test, the higher the best Sharpe you will find by luck alone -- so the top result needs a haircut before you trust it. This article explains the Harvey & Liu multiple-testing adjustment and gives you the intuition to deflate a best-of-many Sharpe. It is educational only, not financial advice, and nothing here concerns or predicts profitability.
The multiple-testing haircut is a downward adjustment applied to a strategy's Sharpe ratio to account for how many strategies were tried before that one was selected. The core idea, formalized for finance by Campbell Harvey and Yan Liu, is simple: if you test one strategy, a high Sharpe is mildly interesting; if you test 500 and report only the best, a high Sharpe is close to guaranteed even when every strategy is genuinely worthless. The haircut asks, "How impressive is this result given the number of attempts it won?" and shrinks the reported statistic accordingly.
Think of it as a correction for a hidden search. A single backtest reports one number. But the process that produced it -- sweeping parameters, swapping indicators, retuning stops -- is a search over many candidates, and the winner is the maximum of many noisy draws. The maximum of many draws is systematically larger than any single draw, so the raw Sharpe of the survivor overstates the underlying signal. The haircut is not a penalty for effort; it is an honest accounting of selection. This is the same integrity-first lens behind our free [Backtest Health Check](/backtest) and [Overfitting Check](/backtest/overfitting): the question is never how good the equity curve looks, but how much of that appearance survives correction. For the broader failure taxonomy, see [why most backtests fail](/learn/why-most-backtests-fail).
Because the best of many noisy results is a biased estimate of skill. Suppose every strategy you test has zero true edge -- pure noise. Each backtest still produces some Sharpe estimate that is scattered around zero by sampling variability. Test one, and you expect roughly zero. Test 500 independent noise strategies, and the maximum observed Sharpe is no longer near zero; it sits well into the positive tail, purely because you took the largest of 500 random draws.
The rough intuition: the expected maximum of N independent standard-normal draws grows with the square root of the logarithm of N. It grows slowly, but it grows relentlessly. Going from 1 to 100 trials, then 100 to 10,000, each step nudges the expected best higher. So a Sharpe of, say, 1.5 that would be striking from a single pre-registered test can be entirely unremarkable as the champion of a large sweep. Worse, real strategy sweeps are correlated -- variants share data and logic -- which complicates the exact count but does not remove the inflation. The takeaway is uncomfortable but clean: the number of things you tried is part of your result, and omitting it makes the result look better than it is. None of this speaks to whether a strategy makes money; it speaks only to whether the observed number reflects signal or search.
A Bonferroni-style adjustment controls the chance that any of your N tests looks significant by luck. Instead of judging the best strategy against the usual single-test bar, you divide your significance threshold by the number of trials -- so with 500 variants and a target of 0.05, each variant must clear roughly 0.05/500 = 0.0001 to count. Equivalently, you demand a much larger t-statistic (and thus a higher Sharpe) before believing the survivor.
Harvey and Liu translate this into a haircut on the Sharpe ratio itself. Their procedure converts the best strategy's Sharpe into a t-statistic, applies a multiple-testing correction (Bonferroni, or the less conservative Holm and Benjamini-Hochberg-Yekutieli methods they also study), and converts the adjusted t-statistic back into a haircut Sharpe. A worked intuition: a strategy showing Sharpe 1.0 over a modest sample might carry a t-statistic around 2. If it is the winner of 200 trials, the Bonferroni-adjusted threshold is far stricter, and the deflated Sharpe can land well below 1.0 -- sometimes near zero. Harvey and Liu note the haircut is non-linear: already-marginal strategies get cut proportionally harder than strong ones. The exact deflated numbers depend on sample length, trial count, and correlation among trials, so treat any single figure as illustrative, not a promise.
| Trials tried | Bar for 'best' result | Effect on reported Sharpe | |---|---|---| | 1 | Standard single-test | No haircut | | ~20 | Noticeably stricter | Moderate haircut | | ~500 | Much stricter | Large haircut; marginal edges vanish |
The multiple-testing haircut and the Deflated Sharpe Ratio (DSR) are close cousins that attack the same disease from slightly different angles, and using them together is good hygiene. The Harvey-Liu haircut is framed around hypothesis testing: it adjusts significance thresholds for the number of trials and reports a deflated Sharpe under Bonferroni, Holm, or BHY corrections. The DSR, developed by David Bailey and Marcos Lopez de Prado, computes the probability that the observed Sharpe exceeds a benchmark that already accounts for the number of trials, the length of the track record, and non-normality (skew and kurtosis) of returns.
Both share one premise: a Sharpe ratio reported without the trial count is not interpretable. Bailey and Lopez de Prado go further by folding in return shape, because fat tails and negative skew inflate naive Sharpe estimates independently of selection. In practice, run both. If a strategy survives a Bonferroni haircut and still clears a high Deflated Sharpe probability, you have reduced -- not eliminated -- the risk that you are looking at the luckiest of many. We cover the DSR mechanics in [the deflated Sharpe ratio](/learn/deflated-sharpe-ratio), and the underlying overfitting mechanism in [overfitting and curve-fitting explained](/learn/overfitting-curve-fitting-explained). Neither test tells you a strategy will be profitable; both tell you whether a number deserves your attention or your skepticism.
The hardest part of the haircut is counting N -- the number of trials -- honestly, because most of the search is invisible. Every parameter you swept, every indicator you swapped, every stop-loss you retuned, every date range you tried, and every idea a colleague or forum handed you is a trial, whether or not you saved the backtest. Under-counting N is the most common way people quietly cheat the haircut and hand themselves a Sharpe that will not survive contact with new data. A defensible practice is to log the search: keep a tally of variants evaluated per family of ideas, and use that count -- not '1' -- when you deflate.
Once you have an honest N, apply the haircut before you compare strategies, not after you have fallen in love with one. Then corroborate with out-of-sample evidence rather than relying on any single adjustment: hold data back ([in-sample vs out-of-sample](/learn/in-sample-vs-out-of-sample)), roll the window forward ([walk-forward analysis](/learn/walk-forward-analysis)), and stress the path ([Monte Carlo simulation](/learn/monte-carlo-simulation-backtests)). Confirm you have enough trades for any of it to mean anything ([how many trades is enough](/learn/how-many-trades-is-enough)). Our free [Overfitting Check](/backtest/overfitting) is built to surface exactly this selection risk. And remember the frame: the haircut protects the integrity and robustness of your conclusion. It says nothing about future returns, and nothing in this article is financial advice.
Educational only — not financial advice. Trading involves substantial risk of loss.