Survivorship Bias in Backtests: 19 of 21, Then 1 of 15
Survivorship bias is the error of judging a method only on the things that lasted, because the failures dropped out of the sample before anyone looked. In a backtest it inflates results: three swing rules passed on 19 of 21 runs over big winners, and on 1 of 15 runs over laggards.
Survivorship bias is what happens when a sample is filtered by its outcome before it is studied. The losers leave quietly, the winners stay on the list, and any method tested on that list inherits a head start it did not earn.
How it works
The filter runs before the analysis starts. A company that went bankrupt, was bought out or fell out of an index no longer appears in a screener of current stocks. A fund that closed after poor years is merged away and drops out of the fund database.
What remains is the group that made it, and the group that made it is better than the group that started.
Nothing in the data looks wrong. Every price is real and every trade in the simulation happened exactly as the rules said. The error sits in the question the test never asked: which stocks would have been on the list on the first day of the test, with no knowledge of the next ten years?
Backtests are especially exposed because they are usually built from a current list. Testing a rule on “the S&P 500” by downloading today’s members runs it on companies that were, by definition, successful enough to be in the index today. The ones that were removed along the way are missing.
Where the missing losers go
Four common exits remove a stock from easy view: bankruptcy, acquisition, being taken private and removal from an index. The first is the obvious one. The other three are quieter, because the company may still exist and the old ticker simply stops producing new bars.
Free data feeds make it worse. A feed that shows current symbols has no reason to keep a delisted one. On 25 September 2026 the Yahoo Finance chart feed returned an error for WBA, the Walgreens ticker, and no price history at all.
Walgreens stopped trading on Nasdaq when Sycamore Partners completed its takeover on 28 August 2025, according to Sycamore’s own announcement. A test assembled from that feed today would not know Walgreens existed.
Memory is a filter too. The stocks a trader thinks of first are the famous winners, so a rule tried on “a few big names” is tried on survivors without anyone deciding to.
A worked example
Start with a hypothetical, to see the size of the effect. Ten funds launch on the same day. After ten years, six are still open with an average return of +8%. The other four closed after averaging −30% before they shut.
A database of open funds reports the six: an average of +8%. The honest average over all ten is (6 × 8 + 4 × −30) ÷ 10 = (48 − 120) ÷ 10 = −7.2%. The same ten years look 15.2 points better simply because four rows are missing. No figure was changed; four were deleted.
Now a real case from the swing backtests. In August 2026 three dip-buying rules were run in TradingView’s Strategy Tester on seven stocks: AAPL, MSFT, GOOGL, AMZN, NVDA, META and TSLA, from January 2015 to August 2026. The rules were the RSI(2) pullback, the five-day low and buying a 1% gap down.
Each run counted as a pass only if at least 60% of trades closed higher, the profit factor was 1.3 or more and there were 30 or more trades.
Twenty-one runs, nineteen passes. The only two misses were the five-day low on Apple and the gap-down buy on Nvidia. Read on its own, that is a rule set that works almost everywhere.
Why a dip-buying rule is most exposed
Every one of these rules buys weakness and waits for a bounce. On a stock that rose for eleven years, a bounce usually came, so the test was really measuring the stock’s decade. The seven were not picked at random: they were chosen after the fact, as the market’s biggest winners.
A trend filter cannot fix that. All three rules trade only above the 200-day average, which removes the worst stretches of a falling stock. It does nothing about a list in which almost no stock fell for long.
The original data
The same code then ran on five large caps chosen because they had lagged badly: INTC, PYPL, DIS, NKE and WBA, over the same window with the same costs.
Fifteen runs, one pass: the five-day low on PayPal, with a profit factor of 2.368 on 32 trades. That is 19 of 21 on the winners against 1 of 15 on the laggards.
Walgreens barely traded. Drop Walgreens, whose three runs produced 1, 5 and 10 trades, and it is 1 of 12.
Pooled across the three rules, 1,057 of 1,617 trades on the winners closed higher, which is 65.37%. On the laggards it was 378 of 661, or 57.19%. None of the 21 winner runs lost money. Two of the 15 laggard runs did, both on Disney: the five-day low finished −32.17% and the gap-down buy −14.63%.
The profit factor tells the same story in money. Across the 21 winner runs the median was 1.81. Across the 12 laggard runs with enough trades to read, it was 1.10, which after costs is close to break-even. Every run is in the published tables: the per-ticker results and the five-stock retest.
Neither list is fair, and that is stated plainly. Picking five laggards is the mirror image of picking seven winners, so the truth for an ordinary stock sits somewhere between the two results.
What the pair does prove is that the first result was mostly a property of the list. Both lists were also tested on one window only; the out-of-sample split was run separately on one rule and three stocks. These are simulated past results, not a forecast.
When it fails
Correcting for survivorship can overshoot. A test that deliberately loads its list with losers, as the five-stock retest did, is biased the other way. It is useful as a stress test and misleading as an estimate.
The fair version fixes the list on the start date: every stock that met a rule such as “in the index” or “over a set market value” on that day, including the ones that later vanished.
Point-in-time data is harder to get than it sounds. Free feeds rarely keep delisted symbols, and index membership lists usually show today’s members. Without the history of who was in and who was out, a test can only estimate the bias rather than remove it.
Delisting is not always a loss. Walgreens left the market through a buyout, and many takeovers pay holders a premium. Dropping those names can bias a test in either direction, which is why the honest answer is to keep them rather than guess.
It is not the only flattering error. A clean, survivorship-free list still leaves room for overfitting, look-ahead mistakes and understated costs. A test that has fixed one of these has not fixed the others.
And a small sample cannot settle it either way. Twelve usable runs on four stocks show the direction of the effect, not its exact size. A broader, neutral list would be needed to put a reliable number on it, and none has been run yet.
Related
Backtesting covers the full list of errors that make a historical test look better than it was, of which this is one.
Overfitting explains the closely related mistake of tuning a rule until it fits the past. And the RSI(2) strategy page shows one of these three rules in detail, including how it behaved on the five laggards. None of these results is a recommendation to trade.
Before trusting any backtest, ask how the list of stocks was chosen and on what date. If the answer is today’s index, today’s watchlist or whatever has done well, the test has already seen the ending, and the result belongs to the list rather than the rule.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.