How to read a backtest without chasing one big number
A big headline number is usually the least useful part of a backtest. Ask four questions about what is underneath it before trusting any result.
Key takeaways
- Ask how many trades produced the result; a handful of trades is barely a sample.
- Look at the worst dip along the way, not just where the result ended.
- Check the rule on data it was not tuned on.
- Shuffle the order of the same trades to see how much was skill and how much was luck of sequence.
Why one number is not enough
A single headline number, such as a total return, looks like the only thing that matters. It rarely is. It hides how many trades produced it, how rough the ride was, and whether the rule was tuned to the very period it is shown on.
Question one: how many trades?
A result built from four trades over several years is not a track record; it is barely a sample. One unusually good trade can carry a figure that looks impressive and says almost nothing about what happens on the fifth trade.
Question two: how deep was the worst dip?
Drawdown is the worst fall from a peak to a later low. A strategy that ends well but dropped a great deal on the way asks you to sit through that drop first. Many people cannot, and they abandon a strategy at the bottom of a dip it would have recovered from.
Question three: was it tested on unseen data?
A rule tuned again and again on one stretch of history will almost always look good on that exact stretch. The honest test is whether it holds up on a period it never saw. Walk-forward testing does this by tuning on one part of history and testing on the next.
Question four: what if the order changed?
Take the same set of trades and shuffle the order in which they happened. If the story falls apart in some of the shuffled orders, the original result may have been a lucky sequence rather than a repeatable edge. A Monte Carlo test does exactly this.
In SmartPaperDesk
The Advanced tools run both checks, Monte Carlo reshuffling and walk-forward testing, on a strategy you choose. Read the results carefully and treat them as a way to question a rule, not as a forecast.
Questions people ask
What is drawdown?
Drawdown is the fall from a peak in value to the next low. The maximum drawdown is the worst such fall in the test, and it shows how much pain the rule can cause along the way.
What is overfitting?
Overfitting is tuning a rule so tightly to past data that it fits the noise. It looks excellent on that history and usually disappoints on new data.
Explore the Advanced tools on the free plan
Keep reading
SmartPaperDesk is an educational paper-trading tool. Examples use a made-up Stock A.
For educational purposes only. This is not investment advice. Past performance does not guarantee future results.