Quantitative Analysis: Measure, Then Decide
Quantitative analysis is deciding from measurements you have taken rather than from an argument that sounds convincing. The order matters more than the mathematics - ask a question with a checkable answer, decide how you would test it, then get the data right and run the test once.
How it works
Quantitative analysis means deciding from measurements rather than from opinions. A claim becomes a number somebody else could check, and the number decides. It is a discipline about evidence, not a category of strategy — technical analysis can be done this way or by eye.
It starts with a question that has a checkable answer. “Do indicators work” is not one. “On this series, does a close above the twenty-bar high lead to a higher close ten bars later” is.
Most of the work is cleaning data, not modelling it. Missing bars, adjusted against unadjusted prices, mismatched time zones and dead instruments each produce results that look real and are artefacts of the file.
And a simple model on good data beats the reverse. A complicated model on dirty data can always be made to look excellent, and when it fails nobody can say which part did.
How you fool yourself
Search long enough and chance will hand you a result. Run forty tests at a five per cent threshold and two will look significant with no effect present at all. That is arithmetic, not bad luck.
So write down the test before you run it. The question, the threshold, and how many variants you intend to try — because a threshold chosen after seeing the result is not a threshold. That is the line between measurement and overfitting.
And keep data back that the model never touched. Look at the held-out block once. Adjusting between looks turns it into training data, which is why forward testing beats any split of the past.
In practice
A model without costs in it is a description, not a plan. On this site’s shared history a round trip costs 2% of a median bar’s range, so a short-horizon result turns as much on the cost assumption as on the logic.
Participation is a measurable input and mostly unused. Volume sits on nearly every series and price-only models dominate anyway, partly by convention and partly because the data is less consistent across venues.
A longer series is worth more than a cleverer method. Sample size sets what a measurement can resolve at all, so a slow strategy needs decades before its record means anything.
The rare event is under-represented in every sample. An opening gap is the clear case: what matters most is what the history holds least of, so the model is weakest where the damage is largest.
And the model says nothing about where the risk sits. It measures an average across many trades; the stop loss on the trade in front of you is a separate decision, and position sizing is not a modelling question.
Every round trip costs 2% of a bar. Which makes trade frequency a design choice rather than an outcome, and one to measure before the method is built.
A procedure you can actually run
Write the question first, in one sentence, naming the instrument and the horizon. “Does a close above the twenty-bar high lead to a higher close ten bars later” is testable. “Do breakouts work” is not.
Then state the test and the threshold before touching the data. Which measurement decides, what result would count, and how many variants you are allowed. Count the alternatives honestly — every parameter swept and every instrument checked is an attempt.
Then run it once, with costs included, on a block you have not looked at. Report the number of attempts beside the result. A finding from one test and the same finding from the fortieth are different objects, and only the count tells them apart.
And backtesting is the same procedure applied to a whole set of rules rather than a single question. Only the number of moving parts changes.
What quantitative analysis is not
It is not a category of strategy. It is a way of checking one.
It is not mathematics for its own sake. The arithmetic is usually simple.
It is not systematic trading. That is rules, this is evidence.
And it is not protection from being wrong. It only makes wrong findable.
When it fails
In a flat period every measurement loses its power. Inside a trading range the difference you are trying to detect shrinks below the noise, and a method that measured something real in a trend measures nothing here.
The second failure is comparing against the wrong alternative. A result measured against zero rather than the base rate flatters itself, because the drift of the market is doing the work.
A third is the summary statistic that hides the shape. A mean is fine, standard deviation less so on a fat-tailed distribution, and a Sharpe ratio says nothing about the worst day.
A fourth is treating correlation as a mechanism. Two series moving together over a chosen window is a measurement, not an explanation, and the window is usually doing the work.
A fifth is the sample that quietly survived. A history holding only the instruments still listed is the least visible bias in this work, because the failures were deleted before you measured.
And a sixth is re-testing the data you held back. Once it has informed a change it is training data, and the honest response is to say so and find more.
The original data
On this site’s shared 576-bar history the base rate for a higher close ten bars later is 54% across 566
observations, and 52% over one bar across 571 observations. Both figures are in
research/series-measurements.json, produced by site/measure_series.py.
That 54% is the whole method in miniature. A signal advertised as right 55% of the time is a claim about one percentage point over doing nothing at all — and the round trip costs 2% of a median bar’s range, more than the edge being claimed. Measure the alternative before believing the claim.
The corpus figures say the same from the other side. Twelve videos carry “quantitative” in the title at
a median of 204 views, against 211 on “win rate” at a median of 11,527, per
research/broker-coverage.json. Measure the base rate for your instrument and horizon before believing any
claim, then subtract the costs.
Related
Systematic trading is what happens once the measurements become rules precise enough to run without you. Overfitting is the way this work goes wrong, and the reason the test gets written down first. And probability is the language the results are stated in, including why five per cent means less than most people assume.
I used to accept a setting because somebody I respected said it worked. The first time I properly measured one on my own data, the result was so much weaker than the claim that I stopped trusting anything I had not checked myself. What changed my trading was not a better method, it was the habit of asking what happens without the signal. Most claimed edges do not survive that question.
— Michael Whitman
This page is educational, not financial advice. Test every idea on your own charts before risking money.