How to Backtest a Trading Strategy
To backtest a strategy, split the history in two before looking at either half, tune the rule on the first half only, then run it once on the second with real costs included. The held-out half is the only part of the test that means anything.
A backtest is not a measurement of a strategy. It is a measurement of a strategy and the process that chose its settings — and if that process saw the answers, the test says nothing.
Before you start
Charting software with enough history to give you at least a few hundred bars on the timeframe you trade. Free tiers are fine.
A rule already written down, in the sense the algorithmic trading page means: a stranger could apply it and get the same trades you would.
Your real round-trip cost — spread plus commission plus a realistic allowance for slippage. Not the advertised spread.
If any of the three is missing, stop here. A backtest of a vague rule with no costs in it is an exercise that can only produce a flattering number.
The steps
1. Write the rule until it has no judgment left in it
Entry condition, stop, target, and what happens if neither is hit. “Wait for confirmation” is not a rule; “enter on the close above the prior high” is.
2. Split the history in two and write down where
Do this before you look at either half. The split date goes in your notes, because a split chosen after seeing the data is not a split.
3. Tune on the first half only
Try whatever you like here. This half is the one you are allowed to fit.
4. Add your costs to every trade
Deduct the round trip from each one. Say a round trip costs $0.02 and an ordinary bar spans $1: that is 2% of a bar, and across 100 trades it adds up to two whole bars of movement handed over before any result is counted.
5. Run once on the second half
Once. If you adjust anything after seeing it, that half is now tuned too and you have no test left.
6. Run the same trades on a schedule and compare
Take the same number of entries every Nth bar, ignoring the rule. Same stop, same target, same costs. That is your control, and without it a positive result is unreadable.
How to tell it worked
Not by whether it made money — by whether the held-out half agrees with the first.
Here is the kind of test that shows why. Suppose sixty settings of one simple rule are tried on the first half of a price history. The best of them makes +8 points.
Suppose that same setting then makes +5 on the second half and ranks 19th of the 60. The best setting on the second half is a different one entirely, at +11, and nothing in the first half’s ranking could have pointed to it.
Suppose, too, that across all 60 the two halves agree only weakly. That is some information, and not much: a setting that came top on the first half is more likely than average to do well on the second, and nowhere near certain to.
So the test passes when three things are true. The out-of-sample result is positive after costs. It beats the schedule control. And there are enough trades to mean something — 25 trades leaves a win rate uncertain by about 20 percentage points either way, and roughly 400 trades are needed to pin it to five.
What a backtest cannot tell you
Four things survive no test, and knowing them stops you trusting a good result too far.
Your fill. A backtest assumes you got the price on the chart. In a fast market you did not, and the gap between the two is largest exactly when the signal is strongest.
Your size. The test buys any quantity at the printed price. A real order meets the liquidity that was actually there, which on a thin instrument is a different trade entirely.
A gap. The move happens between two bars, so no stop and no reaction speed helps — the gap trading page has what that costs.
And whether you would have taken them. A hundred mechanical trades on a screen is not a hundred trades you sat through, including the seven-week losing run inside them.
None of that is a reason to skip the test. It is a reason to treat a passing backtest as permission to trade the rule small, rather than as evidence about what it will pay.
The original data
Across our study of 24,971 trading videos, 462 have backtest or backtesting in the title. The median one gets 2,234 views, 84% never pass 50,000, and the median length is 12.6 minutes.
The corpus carries description text for 133 of those 462, and across those 133, 10 mention invalidation, failure, or what a bad read looks like.
8%, on the one subject whose entire purpose is finding out whether something is wrong. A field of 462 videos about testing, of which about one described video in thirteen mentions a test failing.
When it fails
The settings and the test share a history
This is the failure, and everything above is one procedure for avoiding it. A result chosen from sixty candidates on the same data it is then judged against is a search result, not a finding.
You re-tuned after a bad stretch
A rule adjusted whenever it disappoints has never been tested at all — it becomes a record of many rules with a few trades each, which is the trading journal failure applied to a spreadsheet.
The costs were left out
A backtest without costs is not optimistic, it is meaningless on any rule that trades often. Costs are the largest single line in the frequency table.
The conditions changed and the rule did not
A rule fitted to a trending stretch keeps firing in a range, and the backtest that produced it never contained one. Check what your test period actually held before trusting what it says.
Related
How to automate a strategy is the next step once a rule survives this, and where the parameter count starts to matter.
How to build a trading plan is where the costs and sample size on this page come from.
And do trading bots work runs this exact test in full, with the grid and the correlation written out.
I decide the split before I open the chart, and I write the date down. Without that, a backtest is really a search for the settings that fit what already happened, and you cannot tell the difference from the inside.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.