WhitmanTrading

Forward Testing: Nothing Can Be Tuned

Forward testing records a method's signals in advance and scores them once the outcome arrives. Because nothing can be adjusted after the fact, it is the only test that cannot be contaminated by hindsight, and it is slow enough that almost nobody completes one.

How it works

A forward test writes down what the method says, before anything happens. Entry price, stop, target, size and reason, recorded at the time the signal appears and scored later.

The difference from a backtest is direction. A backtest runs rules over data that already exists; a forward test waits for data that does not.

That single property removes most of the ways a test can lie. No lookahead, no parameter fitting, no selective sampling — because the rules were fixed before the outcome existed.

The cost of the honesty

It runs in real time, and that is the whole objection to it. A backtest of ten years takes seconds; ten years of forward testing takes ten years.

A strongly rising stretch of an illustrative price series with an account curve that tracks a coin flip, one marked level.
A slow method takes months to reach a usable sample. Illustrative chart - not real market data.

Sample size is the binding constraint. A method producing four signals a week needs about six months to reach a hundred trades, and a hundred is the point at which the result starts to mean something.

The discipline is in the writing, not the waiting. A signal recorded after the bar closed is a recollection, and a stop decided once price has moved is a different stop from the one the method specified.

Skipped setups belong in the record. They are the half of the sample no broker statement contains, and a pattern of skipping in one direction — or after a loss — is a finding available no other way. The gap between the recorded signals and the trades actually taken is often the most valuable output of the whole exercise.

In practice

Score the trades net of cost. A method averaging $30 a trade gross that pays $8 per round trip keeps $22, and a forward test recording gross results has quietly reintroduced the error a backtest is criticized for.

It records conditions as well as outcomes. Volume, session and whether news was scheduled are all available at the time and worth tagging, because sorting a hundred trades by tag produces findings reading them chronologically never will.

Timeframe decides how long the test takes. A daily method reaching a hundred trades is most of a year of patience with no verdict available in the meantime.

It captures gap behavior correctly because the gap actually happens, which is the specific thing most simulators model wrongly.

Planned stop against actual fill is the measurement to keep. The difference is your real slippage, it is measurable, and most traders have never checked theirs.

Costs are where a forward test and a backtest most often diverge. The backtest assumed a number; the forward test can look it up.

Running one without waiting a year

A forward test does not have to be the only thing you are doing. Record the signals in parallel with whatever you are already trading — the recording costs nothing, needs no capital, and produces a clean sample alongside a messy one.

Then compare the two at the end. If the recorded signals did better than the executed trades, the problem is execution and no amount of method-changing will fix it. If they did the same, the method is what it is. That comparison is the reason to run a forward test even when you are confident in the method, and it is unavailable from any other kind of testing.

The record needs one more column than most people put in it: what you expected to happen. A sentence written before entry, saying where price should go and roughly how fast, turns a list of outcomes into a list of predictions — and only predictions can be graded.

Without it, a review reads results and invents reasons for them. Memory assembles an explanation consistent with whatever happened, and the explanation feels like a recollection. The expectation column is the cheapest defense against that and it is the difference between a forward test that teaches something and a trade log that confirms what you already believed.

What forward testing is not

It is not paper trading. That tests mechanics; this tests the method.

It is not a backtest. Nothing in it can be re-run or adjusted.

It is not fast. Real time is the price of the honesty.

And it is not conclusive at twenty trades. Sample size still applies.

How this connects

The failure it is built to catch is overfitting. A backtest tuned in a strategy tester can fit its parameters to noise, and a forward sample is the first data those parameters have never seen.

What it actually measures is expectancy. The hit rate alone says little; the average win and loss alongside it decide whether the method pays for its costs once real fills arrive.

A demo account is the natural place to run one when no capital is involved, as long as the fills and costs are recorded as honestly as they would be live.

Systematic trading makes the test cleaner. When the rules are fully written down, there is no room to reinterpret a signal after the fact, and the same record can later feed trade automation.

The regime problem has names. A mean reversion method and a trend following method will each look broken in the conditions that suit the other, so tagging market structure with every trade matters.

Sample size connects directly to risk of ruin. Position sizing chosen from a short, lucky sample can be far too large for the losing streaks the method will eventually produce.

And the result belongs in a trading plan, alongside the maximum drawdown the method showed and the point at which you would stop trusting it.

When it fails

A test that lands entirely inside one regime says nothing about the method. Three months of range will fail a trend method that is perfectly sound, and the sample is too small to separate the regime from the rules.

The second failure is adjusting mid-test. One change and the sample restarts, which is why methods get tested for two years and never accumulate a sample.

A third is recording only the trades taken. The skipped ones are the diagnostic half.

A fourth is scoring gross. Costs decide the verdict on most short-horizon methods.

And a fifth is abandoning it after a losing run. Twenty trades cannot distinguish an edge from noise, which means a decision made at twenty is made on nothing.

The original data

The S&P 500 shows why every forward test needs a base rate beside it. From 1950 to 2026 the index closed higher on 53.1% of its 19,299 trading days, and 59% of its ten-day windows ended above where they began. A long-only method that calls 55% of days correctly is only about two points above simply assuming up, and a forward test that never sets itself against that base rate cannot tell the two apart.

Regimes explain why short forward tests mislead so reliably. Since 1950, the index’s longest spell below a prior record lasted 1,897 trading days, from 11 Jan 1973 to 17 Jul 1980, so a test of a few months can easily sit inside one dominant condition and produce a verdict about that condition rather than about the method. Decide the sample size before starting and hold to it — a hundred trades scored net of costs, with the skipped setups recorded, is worth more than a year of impressions.

Backtesting is the fast filter and its six failure modes. Paper trading tests the mechanics rather than the method. And trading journal is the record a forward test is written into.

What I actually do

I forward test before I trust a method, because it separates the method from the execution. Record the signal in advance, then compare it against the trade you actually take. The gap between those two is usually where the real problem lives.

— Michael Whitman

This page is educational, not financial advice. Test every idea on your own charts before risking money.