Backtesting That Doesn't Lie
Survivorship bias, look-ahead bias, slippage modeling, gas accounting, failed transactions, and the tools (Hummingbot, Jesse, vectorbt, LEAN) and data sources (Kaiko, Dune, Allium) that make honest backtests possible.
35 min · expert · part of Trading Algorithms & Market Microstructure
What you'll learn
- Why Most Crypto Backtests Are Wrong
- Survivorship Bias: Delisted Tokens and Dead Pools
- Look-Ahead Bias: Data You Could Not Have Had
- Slippage, Gas, and Failed Transactions
- Tooling: Hummingbot, Jesse, vectorbt, LEAN, CCXT
- Data Sources: Kaiko, Dune, Allium, Tardis
Key terms
- Survivorship bias
- A data error where the universe of instruments is restricted to those that survived to the end of the period. In crypto this is severe because the failure rate of tokens is high and most data providers do not retain dead tokens.
- Look-ahead bias
- Using information in a backtest that would not have been available at the moment the strategy supposedly acted. Most common with block timestamps, daily closes, and adjusted historical data.
- Slippage
- The difference between expected execution price and realized execution price. In on-chain trading slippage has multiple components: price impact, MEV exposure, latency drift, and protocol fees.
- Hummingbot
- Open-source market-making and arbitrage framework supporting both CEX and DEX connectors. Common choice for inventory-aware market making strategies.
- vectorbt
- A vectorized Python backtesting library that uses NumPy operations for dramatic speedups, ideal for parameter sweeps and Monte Carlo testing.
- QuantConnect LEAN
- Open-source institutional-grade C# backtesting and live-trading platform supporting equities, futures, options, and crypto with rigorous handling of corporate actions and survivorship-bias-free datasets.
- CCXT
- A unified Python and JavaScript API client supporting 100+ centralized exchanges. Standard library for CEX live execution.
- Kaiko
- Institutional-grade market data provider covering 100+ CEXs and growing DEX coverage, with full order book snapshots and consolidated reference rates.
- Dune Analytics
- SQL interface on indexed on-chain data across many chains. Standard tool for on-chain historical research and protocol-level reconstruction.
- Allium
- Newer competitor to Dune providing indexed on-chain data with stronger low-latency feeds and institutional reliability.
Read the full lesson in the CryptoBipto app.
Open lessonEducational only — not financial advice.