Product

Validation

Before a signal is trusted it has to beat plain buy-and-hold on data it was never fit on, after costs, after accounting for how many things were tried. So far, nothing has.

The Sharpe ratios in the film show how the gate reads, not live results. The verdict is real: no signal has been certified.

Out of sample, or it does not count.

The best honest annualised result the harness has measured out of sample is about 10.8%. In its own tests nothing has beaten simply holding. We would rather tell you that than sell you a backtest.

Data it was never fit on.

Every signal is fit on one stretch of history and judged on a later one it never saw. Most look brilliant on the first and ordinary on the second. That gap is what Dev, the out-of-sample tester, watches for.

Momentum, 12-1 fit · then test
SignalBuy-and-holdfit | test, rebased at the split

Deflated for everything we tried.

Try enough ideas and one will look good by luck. Priya, the multiple-testing statistician, discounts every result for how many things were tried before it. A Sharpe that survives that is worth a look. None has yet.

PR
PriyaMultiple-Testing ControlFails
Out-of-sample Sharpe 0.58 after 41 variants tried. Deflated, that is 0.14. Not distinguishable from luck.
DE
DevOut-of-Sample TestingWary
Backtest 1.52, live-forward 0.58. The gap is the story.
EL
ElenaParameter StabilityWary
The best lookback jumps from 40 to 90 days between periods. Not stable.

What changes if one passes.

Less than you might think. A validated signal can carry more weight in the floor's argument. It still goes through every limit and veto, and on a client desk it still waits for your click.

If a signal is validated
YesResearch can cite it as evidence in a plan
NoIt cannot skip a limit or outvote a veto
NoIt cannot place an order without your click
YouYou hear about it, with the test, before it is used

A small number of clients at a time.

Tell us a little about you. We reply personally, usually within a few days. If it is a fit, the questionnaire comes next, and your account after that.

Already a client? Sign in to get your install line