Survivorship Bias in Crypto Backtests: The Coins That Aren't There Anymore
Jonny Bravo
-Survivorship Bias in Crypto Backtests: The Coins That Aren't There Anymore
Here's a backtest almost everyone has run without noticing what it assumes. Take the top 50 coins by volume — today's top 50 — pull their full price history, and test a momentum strategy across the basket. The results look great. Momentum in crypto! Well documented!
Now look at what the universe construction did. Every coin in the test is a coin that survived long enough to be on today's list. The ones that were delisted, rugged, sanctioned, or bled to irrelevance — the exact assets a real strategy would also have held on the way down — are not in the study at all. You didn't backtest "momentum on crypto." You backtested "momentum on the coins that turned out fine," which is a strategy you cannot actually run, because you don't know the future survivors in advance.
Equity quants have fought this bias for decades with survivorship-free databases. Crypto makes it worse in three ways:
- Delisting is routine, not rare. Venues cull pairs constantly — low volume, regulatory pressure, a project quietly dying. A universe that looks stable over a quarter loses real members over a year.
- The tail is fat and the floor is zero. A delisted stock often gets acquired or migrates exchanges. A delisted perp's terminal chapter is frequently a one-way trip toward illiquidity — precisely the returns your test silently excludes.
- Data vendors help you fail. Most APIs serve current listings by default. The lazy path — fetch what's listed, backtest on it — is the biased path.
What honest looks like: model the delisting, don't erase it
The fix has two halves. The first is universe construction: build the tradable set as it stood at each point in time, not as it stands today, so the doomed coins are in the test while they were still plausible holdings. Our data layer keeps history for symbols that later vanished, and the point-in-time snapshot discipline means a strategy at time t can't peek at which of them survive.
The second half is what happens when a held asset actually leaves the venue mid-window. Ignoring it (position quietly evaporates), or holding it at its last price forever (a zombie position propping up NAV) are both fictions. The engine's delisting handling is deliberately unglamorous: when an instrument's data ends because the asset left the venue, the position is force-closed at its last tradable price, the exit hits the equity curve, and the event is surfaced in the run report instead of buried. If that final print was ugly, your backtest eats it — because live, you would have.
This isn't only a backtesting concern. In production the same problem arrives as an ops emergency: an asset your live book holds stops being tradable. Our platform runs a delisting monitor that flags active assets that have disappeared from a venue's tradable set — because the honest answer to "what happens when a coin dies?" has to exist on both sides of the research/live divide.
How much does it matter? More than your alpha
The insidious thing about survivorship bias is that it doesn't add noise — it adds signed error, always in the flattering direction. Filtering to survivors systematically removes negative returns from the sample. Studies in equities put the bias at a percent or two of annual return; in crypto, where the delisted tail routinely does -90% before it goes, a survivors-only altcoin study can easily overstate returns by more than the strategy's entire claimed edge.
Which leads to the diligence question worth stealing, whether you're allocating to a manager or about to trust your own track record: "How does your backtest treat coins that were delisted during the test window?" The answers sort platforms instantly. Silence means the bias is in the numbers. A hand-wave means it's in the numbers. "Positions are force-liquidated at the last tradable print and the events are in the report" means someone built the boring machinery that makes the pretty numbers mean something.
Dead coins tell no tales — unless your data layer kept them. Ours did, on purpose.
Backtest against history that still contains the losers — point-in-time universes, delistings force-closed, events on the report. Test a strategy honestly →
Related reading