Imagine you want to know how much you would have made by buying a little bit of all of Wall Street in 2000 and forgetting about it for twenty-six years. You open your platform, load the stocks on the list, run the backtest and get a beautiful equity curve: your money multiplied by twelve.
The problem is that the list you loaded is today's. And most of the companies that existed in 2000 are not on it, because they no longer exist.
We measured it with real data. The result surprised even us.
The mistake of only looking at the survivors
During World War II, the US military studied the bullet holes on bombers returning from missions, to reinforce the most damaged areas. The statistician Abraham Wald pointed out the flaw: they needed to armour precisely the areas without holes. The planes hit there had not come back to tell the tale. Curious, isn't it?
The stock market works exactly the same way. If you only study the companies that are still listed, you are studying the planes that came back. The ones that went bankrupt, were taken over or were kicked out by the regulator vanish from today's data, and a large part of the story vanishes with them. This is called survivorship bias, and it is probably the most expensive and the quietest mistake in backtesting.
The data: 7,532 stocks on 3 January 2000
To measure it we used AniQuant's stock catalogue, which includes the 19,722 companies that have traded on the NYSE, Nasdaq and NYSE American since the late nineties. 13,855 of them are no longer listed: 70%. Yes, you read that right: 70% of them have disappeared.
From that catalogue we took every stock that was trading on the first market day of 2000. That is 7,532 of them (4,702 on the Nasdaq, 2,301 on the NYSE and 525 on NYSE American), and we followed them year by year until the end of 2025.
The curve leaves no room for doubt. By the end of 2007, fewer than half were left, before the financial crisis even started. Today 1,436 are still listed: barely one in five.
And yet Wall Street has not emptied out. Today 5,867 stocks are listed on those three exchanges, fewer than in 2000, but nowhere near one fifth. The reason is that the market renews itself constantly: while some companies disappear, others go public. Of the stocks listed today, three out of four went public after 2000.
That is why the total number is misleading. Today's market and the one of twenty-six years ago share a name, but little else: a list of stocks "as of today" looks nothing like the one an investor had in front of them in 2000. And if you run your backtest starting from the stocks listed today, you are handing your strategy a list of winners that nobody could have known back then.
Not all of them died the same way
A company disappearing does not mean its shareholders lost everything. Many were bought by other companies, and their shareholders got paid, often at a premium. That is why the causes matter:
Almost two out of three were acquired or merged. But 1,266 went bankrupt or were liquidated, and another 311 were removed by the regulator. In total, 1,392 of the vanished stocks had lost more than 90% of their value by the time they stopped trading.
Those are exactly the ones a survivorship-biased backtest never sees.
The same backtest, two results
Back to the portfolio from the beginning: the same amount of money in each of the 7,532 stocks on 3 January 2000, buy and hold. We measured it in two ways:
- Reality: every stock. When one stops trading, its money stays at the last price.
- The biased backtest: only the 1,436 still listed today, which is what you get if you start from today's list of stocks.
| Reality (every stock) | Biased backtest (survivors only) | |
|---|---|---|
| Money multiplied by | ×3.9 | ×12.1 |
| Annual return | 5.4% | 10.1% |
| The typical stock (median) | ×1.14 | ×3.76 |
The biased backtest triples the result. And the gap in annual return, almost five points, is larger than the edge most strategies are looking for.
The median tells the most uncomfortable part: the typical stock of 2000 made barely 14% in twenty-six years, dividends included. The average goes up thanks to a few stocks that multiplied by hundreds. If your backtest only sees the survivors, it believes the typical stock almost quadrupled.
Why it happens without you noticing
Almost nobody falls into this on purpose. It comes in with the data:
- Today's lists. Many backtests, most of them I would say, start from the current members of the S&P 500 or the Nasdaq-100. The companies that left the index, or the market, are not there.
- Free data. Free sources usually only offer live stocks. When a ticker disappears, its whole history disappears with it.
- Reused tickers. The same symbol may belong today to a different company from the one using it in 2005, and mixing the two produces absurd results.
- Fundamentals as of today. If you filter on earnings or debt using today's revised figures, rather than the ones published at the time, you add look-ahead bias on top.
How to avoid it
- Use data that includes delisted companies, with their full history up to the last trading day.
- Rebuild the universe on every date. The eligible stocks on day X must be the ones trading on day X, not today's. The same goes for index membership. This point is FUNDAMENTAL.
- Do not delete the ones that disappear. A position in a company that goes bankrupt has to book the loss, not vanish from the calculation.
- Fundamentals by publication date. Only what the market could know on that day. This does not directly affect most backtests, since they only use technical data. However, if your strategy also relies on fundamental data, this point is crucial.
- Be suspicious of anything too pretty. If a stock strategy beats the market by a lot with hardly any drawdowns, the first suspect is the universe of stocks chosen to run the backtest.
How we do it in AniQuant
When we decided to take AniQuant to US stocks, the first requirement was this one: no company is allowed to disappear from the data. The catalogue keeps the delisted stocks and the date and cause of their delisting (bankruptcy, acquisition, merger or regulatory removal). Every stock backtest rebuilds the universe that existed on each date, and fundamentals are applied by publication date, with no look-ahead.
I am not telling you this to sell you anything: I am telling you because, without it, any result from a stock strategy is suspect, ours included.
If you backtest stocks, the next time a strategy comes out spectacular, ask yourself one question: am I studying the planes that came back?
Want backtests without this bias?
AniQuant works with universes that include the companies that disappeared. Try it free for 14 days.
Download AniQuantFounder of AniSoft (software since 1995) and creator of AniQuant. Writes about honest backtesting, strategy validation and the mistakes that cost money.