The strategy made money, but we could not prove it
On history it beat the market and survived real costs. But you tried 18 variants before you found this one — and at that level of searching a similar result could have appeared by itself.
- Enough trades
- History is long enough
- Commission headroomcostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- Costs are noticeablecostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- Sensitive to latency
- Below the search barnumber of trialsEvery run with changed parameters is one more attempt. The more attempts, the higher the bar the result must clear — random rules look good too.How Many Backtests Is Too Many — Why the Answer Depends on How Many Times You Looked
- Unstable across years
- Forward not checked yetforwardThe same frozen rules running on data that did not exist when the hypothesis was registered. Nothing can be retuned against it.Backtest vs Live Trading — Which Part of the Gap Was Already in the Test
Eight checks: three passed, two passed with caveats, two failed, one is still ahead.
Buying the morning range breakout
For the first half hour the market often trades inside a narrow band. The idea is that price leaving that band sets the direction for the rest of the day. If price rises above the high of the first 30 minutes — buy at the open of the next candle. Then wait for one of three: target, stop, or the close of the session.
One trade per day at most. If neither the target nor the stop fires before the close, the position is closed at market. No overnight holds.
- Event
Price rises above the high of the first 30 minutes of the session
high > max(high[09:30..10:00]) - Filter
Normal and high volatility days only
ATR(14) ≥ 40th percentile over 60 days - Entry
Buy at the open of the candle after the signal
no look-ahead into the current bar - Stop
−0.4 % from entry
checked against the low of every candle - Target
+0.8 % from entry
checked against the high of every candle - Exit
Target, stop or the 16:00 ET session close — whichever comes first
- Size
1 contract, no scaling in and no compounding
fixed size over the whole period - Direction
Long only
the mirror version for shorts was not tested
Rules in canonical form
- rules hash
- 7f3a91c4e08drestores the strategy unambiguously
- parameters
- 4range window (30 min), stop (0.4 %), target (0.8 %), filter threshold (40th percentile)
- free parameters per trade
- 103412 trades / 4 parameters = 103 observations per parameter
You searched too long to believe the find
Every new variant you try raises the chance of hitting a good curve by accident. Eighteen attempts is already a lot: the best of eighteen meaningless rules would look roughly like this. To separate your strategy from that coincidence, it had to show noticeably more.
How this was counted
- metricSharpe ratioExcess return per unit of volatility. On its own it says nothing about how many rule variants you tried before this number appeared.What Is the Sharpe Ratio — and Why a Pretty Number Can Be Noise
- deflated Sharpe = 0.94threshold 1.47
- trialsnumber of trialsEvery run with changed parameters is one more attempt. The more attempts, the higher the bar the result must clear — random rules look good too.How Many Backtests Is Too Many — Why the Answer Depends on How Many Times You Looked
- 18 raw → 11 effectiveeigenvalues of the correlation matrix of variants
- threshold formulanumber of trialsEvery run with changed parameters is one more attempt. The more attempts, the higher the bar the result must clear — random rules look good too.How Many Backtests Is Too Many — Why the Answer Depends on How Many Times You Looked
- E[max SR] + 1.645σE[max SR] ≈ σ·√(2·ln N_eff)
- block permutation test
- p = 0.087913 of 1000 shuffles were worse
Over four years the account would have grown about one and a half times
After every commission and slippage. For comparison: simply buying and holding for the same four years would have grown the account less — but the gap is not as big as the picture makes it look. Everything the strategy added on top of plain buying is about $5,300 a year on a $100,000 account.
Capital and drawdown
January 2021 — December 2024 · start 100In numbers
- net return
- +11.8 % a year+56.2 % over the whole period
- benchmark
- buy & hold +34.9 %SR 0.55 · alpha ≈ 5.3 pts a year
- trades
- 41246 % winners · profit factor 1.24
- time in market
- 6.8 %47 minutes in a trade on average
You would have spent a year below your own peak
The worst moment was May 2022: the account fell 14.7 % from its peak. On a $100,000 account that is minus $14,700. And it took a full year to get back to the previous high. Those are exactly the months when most people close a strategy by hand — and then the promised 11.8 % a year simply never arrives.
In numbers
- max drawdownmax drawdownThe deepest fall from a peak to the next trough. It is the metric people quit on, and a backtest only shows the drawdowns its sample happened to contain.What Is Max Drawdown — and Why Your Backtest Lowballs It
- 14.7 %peak 05.2022 → trough 09.2022 → recovery 05.2023
- the benchmark
- 14.6 %over the same period — comparable risk, only the return differs
- worst month
- −7.4 %best month +8.7 %
- longest losing streak
- 9 trades
Almost a third of the earnings goes to the broker
Before commissions the strategy returned 16.7 % a year, after — 11.8 %. A gap of 4.9 points, or about $4,900 a year from your account. There is still headroom: the strategy only breaks even at a commission 2.6 times yours. But switch to a more expensive broker or a less liquid instrument and the headroom is gone.
In numbers
- gross → costs → netcostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- 16.7 % → 4.9 pts → 11.8 %costs ate 29 % of the gross profit
- critical commissioncostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- $5.80 per round tripagainst your $2.25 — headroom 2.6× (the gate wants 1.5×)
- stress: costs ×2
- +5.9 %a year instead of 11.8
- stress: enter one bar laterSharpe ratioEntry five minutes after the signal: sensitivity to latency.Excess return per unit of volatility. On its own it says nothing about how many rule variants you tried before this number appeared.What Is the Sharpe Ratio — and Why a Pretty Number Can Be Noise
- 0.71was 0.94 — a drop of a quarter
Almost all of the profit sits in two years out of four
2021 and 2023 together gave +56 %, while 2022 was negative and 2024 flat. A strategy whose profit is concentrated in a couple of lucky years can be a memory of those years rather than a rule of the market. On history you cannot tell the two apart — only time can.
In numbers
- profitable years
- 3 of 4but 93 % of the result comes from two of them
- by market regime
- high volatility +0.142 %normal day +0.071 % · low volatility +0.019 %
- PBOholdoutA slice of history sealed at registration. It stays unopened until the verdict, so the rules cannot be tuned to it.Out-of-Sample Testing That Is Actually Unseen — Holdout, Walk-Forward, Purge and Embargo
- 0.31the chance that the variant you picked is below average out of sample
- MinTRL
- 1.58 yearsagainst the 1.67 you have — the history barely covers it
A rejection is a verdict on the evidence, not on the idea
We are not saying the strategy does not work. We are saying that with this data and this many attempts it cannot be proven. You have three exits, and each one has an honest price.
What would change the verdict
- at 2 trials instead of 18number of trialsEvery run with changed parameters is one more attempt. The more attempts, the higher the bar the result must clear — random rules look good too.How Many Backtests Is Too Many — Why the Answer Depends on How Many Times You Looked
- threshold 0.81the result would have passed
- at SR 1.47 instead of 0.94Sharpe ratioExcess return per unit of volatility. On its own it says nothing about how many rule variants you tried before this number appeared.What Is the Sharpe Ratio — and Why a Pretty Number Can Be Noise
- would passat the current level of searching
- confirmed by forwardforwardThe same frozen rules running on data that did not exist when the hypothesis was registered. Nothing can be retuned against it.Backtest vs Live Trading — Which Part of the Gap Was Already in the Test
- the verdict stops depending on Nin 1.6 years
What it was computed on
The calculation runs on a snapshot of history fixed at the moment the hypothesis was registered. Data has grown since — but this report neither changed nor will change: recomputing a year from now gives the same numbers.
- Instrument
- ES — E-mini futures on the S&P 500 index, CMEcontinuous contract
- Stitching
- Roll five days before expiry, difference back-adjustedroll spread included in the costs
- Period
- 04.01.2021 — 12.12.20243 years 11 months · 987 trading sessions
- Bars
- 5-minute, regular session only 09:30–16:00 ET77,025 bars · pre- and post-market excluded
- Gaps
- 0.02 % of bars are missing — shortened holiday sessionsmissing, not filled in — interpolation is forbidden by the engine
- Quality
- Checked for zero volumes, price outliers beyond 10σ and breaks in bar numberingno anomalies found
- Source
- Licensed market data vendorsnapshot snap-2024-12-14 · sha256 a91c…4e08
How every trade was computed
Every rule below is pessimistic by default. Where the outcome is ambiguous, the engine picks the version that is worst for you. Rejecting a good idea beats missing a bad one.
- Signal
- Computed on closed candles onlyindicators never see the current bar — look-ahead is impossible by construction
- Fill
- At the open of the next candlenever at the close of the signal candle
- CommissioncostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- $2.25 per round tripexchange and clearing fees + broker
- SlippagecostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- 0.25 ticks per side, i.e. $12.50 per round tripfixed, independent of size
- Stop and target inside one candle
- If both levels are touched inside one candle, the stop countsworst outcome, 11 such cases out of 412
- Partial fills
- Not modelled: a 1-contract position always fills completelyan assumption in your favour, flagged deliberately
- Margin
- Checked on every bar; no margin call occurred over the period
- Determinism
- The same snapshot and the same rules hash give a bit-identical resultthe random generator is used nowhere except the permutation test, where the seed is recorded
What is an assumption here, not a fact
- slippage is fixed — on a thin market the real one will be higher
- fills are always complete — for one contract on ES that is close to the truth, for size it is not
- latency is not modelled — checked separately by the “enter one bar later” stress: SR falls to 0.71
This report can be re-verified
Everything needed to repeat the calculation exactly a year from now is written below. If the recomputation does not match bit for bit, the error is ours and we want to know.
- data snapshot
- snap-2024-12-14
- rules hash
- 7f3a91c4e08d
- training
- 2021-01-04 — 2023-06-30
- validation
- 2023-07-06 — 2024-06-30
- holdoutholdoutOpened exactly once — at the verdict.A slice of history sealed at registration. It stays unopened until the verdict, so the rules cannot be tuned to it.Out-of-Sample Testing That Is Actually Unseen — Holdout, Walk-Forward, Purge and Embargo
- 2024-07-08 — 2024-12-12 · opened
- forwardforwardRules frozen on 14.12.2024.The same frozen rules running on data that did not exist when the hypothesis was registered. Nothing can be retuned against it.Backtest vs Live Trading — Which Part of the Gap Was Already in the Test
- since 2024-12-14 · running
- trialsnumber of trialsThe counter never resets and cannot be cleared on request.Every run with changed parameters is one more attempt. The more attempts, the higher the bar the result must clear — random rules look good too.How Many Backtests Is Too Many — Why the Answer Depends on How Many Times You Looked
- 18 raw / 11 effective
- cost modelcostsCommission, spread, slippage and financing — subtracted before any metric is computed, so the report shows the curve you would actually live with.Backtest Costs — Commission, Slippage and the Fill Model That Decides the Verdict
- $2.25 per round trip + 0.25 ticks
Check your own idea
The same rules, the same trial counter and the same verdict — on your data. The report is assembled automatically and you can read it for free.
Check an ideaSample report A sample report. The numbers are consistent with each other but were produced on demonstration data — this is not the result of a real test.