How to Backtest a Trading Strategy
To backtest a strategy, split the history in two before looking at either half, tune the rule on the first half only, then run it once on the second with real costs included. The held-out half is the only part of the test that means anything.
A backtest is not a measurement of a strategy. It is a measurement of a strategy and the process that chose its settings — and if that process saw the answers, the test says nothing.
Before you start
Charting software with enough history to give you at least a few hundred bars on the timeframe you trade. Free tiers are fine.
A rule already written down, in the sense the algorithmic trading page means: a stranger could apply it and get the same trades you would.
Your real round-trip cost — spread plus commission plus a realistic allowance for slippage. Not the advertised spread.
If any of the three is missing, stop here. A backtest of a vague rule with no costs in it is an exercise that can only produce a flattering number.
The steps
1. Write the rule until it has no judgement left in it
Entry condition, stop, target, and what happens if neither is hit. “Wait for confirmation” is not a rule; “enter on the close above the prior high” is.
2. Split the history in two and write down where
Do this before you look at either half. The split date goes in your notes, because a split chosen after seeing the data is not a split.
3. Tune on the first half only
Try whatever you like here. This half is the one you are allowed to fit.
4. Add your costs to every trade
Deduct the round trip from each one. On this history a 0.02 cost is about 2% of a typical bar — and across 123 trades it took 67% of the gross.
5. Run once on the second half
Once. If you adjust anything after seeing it, that half is now tuned too and you have no test left.
6. Run the same trades on a schedule and compare
Take the same number of entries every Nth bar, ignoring the rule. Same stop, same target, same costs. That is your control, and without it a positive result is unreadable.
How to tell it worked
Not by whether it made money — by whether the held-out half agrees with the first.
Here is the measurement that shows why. Sixty settings of one simple rule were tried on the first 288 bars of a history. The best made +7.53.
That same setting on the next 288 bars made +5.85 — and ranked 19th of the 60. The best setting on the second half was a different one entirely, at +11.22.
Across all 60, the two halves agree at r = +0.38. Some information, and not much: a setting that came top on the first half is more likely than average to do well on the second, and nowhere near certain to.
So the test passes when three things are true. The out-of-sample result is positive after costs. It beats the schedule control. And there are enough trades to mean something — 25 leaves a win rate uncertain by about 20 percentage points either way, and roughly 400 are needed to pin it to five.
What a backtest cannot tell you
Four things survive no test, and knowing them stops you trusting a good result too far.
Your fill. A backtest assumes you got the price on the chart. In a fast market you did not, and the gap between the two is largest exactly when the signal is strongest.
Your size. The test buys any quantity at the printed price. A real order meets the liquidity that was actually there, which on a thin instrument is a different trade entirely.
A gap. The move happens between two bars, so no stop and no reaction speed helps — the gap trading page has what that costs.
And whether you would have taken them. A hundred mechanical trades on a screen is not a hundred trades you sat through, including the seven-week losing run inside them.
None of that is a reason to skip the test. It is a reason to treat a passing backtest as permission to trade the rule small, rather than as evidence about what it will pay.
The original data
Across our study of 24,971 trading videos, 812 cover backtesting. The median one gets 2,115 views, 86% never pass 50,000, and the median length is 13.3 minutes.
The corpus carries description text for 150 of those 812, and across those 150, 11 mention invalidation, failure, or what a bad read looks like.
7%, on the one subject whose entire purpose is finding out whether something is wrong. A field of 812 videos about testing, of which about one in fourteen mentions a test failing.
When it fails
The settings and the test share a history
This is the failure, and everything above is one procedure for avoiding it. A result chosen from sixty candidates on the same data it is then judged against is a search result, not a finding.
You re-tuned after a bad stretch
A rule adjusted whenever it disappoints has never been tested at all — it becomes a record of many rules with a few trades each, which is the trading journal failure applied to a spreadsheet.
The costs were left out
A backtest without costs is not optimistic, it is meaningless on any rule that trades often. Costs are the largest single line in the frequency table.
The conditions changed and the rule did not
A rule fitted to a trending stretch keeps firing in a range, and the backtest that produced it never contained one. Check what your test period actually held before trusting what it says.
Related
How to automate a strategy is the next step once a rule survives this, and where the parameter count starts to matter.
How to build a trading plan is where the costs and sample size on this page come from.
And do trading bots work runs this exact test in full, with the grid and the correlation written out.
The habit that changed my results was deciding the split before opening the chart, and writing the date down. Every backtest I ran before that was really a search for the settings that fitted what had already happened, and I could not tell the difference from the inside.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.