Backtesting: The Curve Is the Seduction
Backtesting runs a set of rules over historical data to produce the equity curve those rules would have generated. The output is only as good as the assumptions about fills, costs and data quality, and six specific errors reliably make a backtest look better than the live result.
Covered on this page: TradingView.
How it works
A backtest applies rules to past data and records the results. Entry, exit, size and cost are specified, the software walks the history, and an equity curve comes out.
The curve is persuasive out of proportion to its evidence. A rising line looks like proof, and it is really a statement about how the rules interacted with one particular history under a set of assumptions you chose.
The six errors
Sample size is the first and most common. Twenty trades is not enough to distinguish a method with an edge from one without; two hundred begins to be. Most published backtests run on far fewer.
Cost understatement is the second. Say a method’s hypothetical edge is 0.06% a trade and its true round trip costs 0.08%: a test applying half that cost, 0.04%, turns a losing method into a winning one on paper.
Fill assumptions are the third. A resting limit order fills in a backtest whenever price touches its level; in reality it joins a queue. A stop fills at its trigger; in reality it becomes a market order.
Lookahead bias is the fourth and the most damaging. Using a bar’s close to make a decision at that bar’s open, or a revised data point that was published later, produces results that were never available.
Survivorship bias is the fifth. A test on today’s index members excludes every company that was removed, which is precisely the set that would have hurt. And the sixth is overfitting — tuning parameters until the curve looks right, which deserves its own page.
In practice
Model costs before anything else. A method that survives a pessimistic cost assumption is worth testing further; one that only works at optimistic costs has already failed.
Market impact is assumed away entirely. In thin volume a real order moves price and a simulated one does not, which flatters every result taken in quiet conditions.
Slow methods are the hardest to test honestly. Two hundred daily-chart trades can require a decade of data, across regimes that no longer resemble each other.
Many engines fill a stop at its level across a gap. That single modeling error removes the largest risk in overnight trading from the record.
A stop needs slippage attached. Adding a fixed pessimistic amount to every stop fill is crude and far closer to reality than assuming none.
High-frequency results are the most fragile. The more trades a method takes, the more of its result is determined by the cost assumption rather than by the rules.
Test against the base rate
A backtest that beats zero has proved nothing. The comparison that matters is against what would have happened anyway: on the S&P 500 from 1950 to 2026, 53.1% of trading days closed higher than the day before.
That number is the bar every entry rule has to clear. A pattern with a 55% next-bar hit rate sounds useful and is about two percentage points above doing nothing at all — before costs, which on a per-bar edge that small will usually consume the entire difference. Compute the base rate first and put it beside every result; it converts most impressive-looking hit rates into unremarkable ones, which is exactly what it should do.
One structural habit prevents most of these errors at once: hold data back. Fit on the first two thirds of the history, then run once on the final third and accept whatever comes out. The reserved section is only useful while it remains untouched, so it can be used once — a second look at it turns it into part of the training set.
The discipline is harder than it sounds because a poor out-of-sample result invites one more adjustment. That adjustment is the moment the test stops being a test. Write down the result before changing anything, so the record shows what the method did rather than what the third version of it did.
What a backtest is not
It is not evidence of a future edge. It describes one history.
It is not a live result. Fills and costs are modeled, not paid.
It is not objective. Every assumption in it was chosen.
And it is not enough on its own. Forward testing is the check.
How this connects
A backtest turns a rule into a number, which is the only honest way to argue about whether a rule works.
It requires the rule to be unambiguous. Entry, exit, position sizing and invalidation all specified - and most discretionary ideas fall apart at exactly that step.
Costs decide the answer. The bid-ask spread and slippage are paid on every round trip, and a test run without them describes a market nobody trades in.
Overfitting is the characteristic failure. Enough parameters will always describe a sample well, which is why forward testing on data you have not looked at is the number that counts.
And a strategy tester is just the software that does this, so everything true here applies to whichever one you run.
When it fails
A period dominated by one regime produces a misleading verdict either way. A trend method tested across a range looks broken; the same method tested across a trending year looks excellent. Neither is a property of the method.
The second failure is testing on the data you tuned on. The result is a description of the tuning.
A third is a survivorship-contaminated universe. The failures are the missing evidence.
A fourth is ignoring the sequence. Two curves with the same total return and different orderings are very different to hold.
A fifth is stopping at the backtest. It is the cheap first filter, not the decision.
And a sixth is testing on a price series that was adjusted after the event. Splits and dividends are applied backward, so the history the software reads today is not the history that printed at the time.
Any rule keyed to a round number, a dollar threshold or a minimum gap size is therefore checked against prices nobody ever saw on a screen, and the test will never report that as a problem.
The original data
How much does a long-only rule get for nothing? On the S&P 500 from 1950 to 2026, 53.1% of days closed up, and across 19,290 ten-day windows 59% finished higher. Any long-only backtest has to be read against those rates, not against zero, and after its costs.
The sample problem is easier to see with dates attached. Since 1950, the S&P 500’s longest spell below a record ran from January 1973 to July 1980, so a test covering only those years would never see a new high, and no conclusion drawn from one regime is worth acting on. Before reading any backtest result, find the number of trades behind it — if it is under a hundred, the curve is a story about a handful of events, however convincing the line looks.
Related
Forward testing is the check a backtest cannot perform on itself. Overfitting is the error that survives every other fix. And strategy tester covers the software and what its defaults assume.
One line can turn a backtest into fiction — for example, an exit that uses the bar’s closing price to decide something at the open. The whole result looks excellent and none of it was tradeable. Write the data-availability check before you write the strategy.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money. Some links on this page earn a commission if you buy through them. It costs you nothing and it does not decide what appears here or in what order — how these pages are made is set out in our methodology.