Open source · Deflated Sharpe + sealed holdout
Your backtest is
probably lying to you.
Search hard enough over historical data and you'll always find a beautiful Sharpe — by pure luck. OverfitGuard gives any strategy an honest verdict, and refuses to bless what it can't defend.
$ pip install overfitguard
The one idea
A high in-sample Sharpe is not evidence. It's what a big enough search always produces. Evidence is what survives data it has never seen — after honest costs, and after you've owned up to how many things you tried.
// OverfitGuard treats a great in-sample number as expected, not as proof.
Three defenses — two of them independent
The discipline that separates an edge from a mirage.
Deflated Sharpe Ratio
Penalizes your Sharpe for how many strategies you tried, how short your record is, and how fat your tails are. If a search that wide would throw up a number this good by luck, it says so.
Bailey & López de Prado, 2014Sealed out-of-sample holdout
The tail of your track record is locked away and never touched until judgment. It catches the regime-fitting the deflation math alone can't — the classic “great until it wasn't.”
The test the math can't fakeWhite's Reality Check
Judges a whole search at once: was the best of your 500 configurations real, or just the luckiest of 500 coin-flips? A moving-block bootstrap answers honestly.
Sullivan, Timmermann & WhiteIt says no — and sometimes yes
Two real reports. Open them.
Not a mockup. Each is generated by the tool and swings on the evidence, so the verdict means something.
A caught mirage
A backtest that looked like a discovery — and wasn't. The edge reverses the moment the search runs out of data to fit.
A genuine edge
Survives deflation for the trials tried and holds up out-of-sample. Proof the tool is calibrated, not just a naysayer.
Both regenerate from one command: overfitguard validate returns.csv --trials 500 --html report.html
The companion course — free
Fooled by Backtests
A short, honest course that teaches these defenses by using the tool on real data — including a research program that rigorously tested a dozen strategy families and found buy-and-hold beat essentially all of them.
It ends with hands-on labs and a capstone: audit your own strategy in one command. The moment you run it on a backtest you believed in is the moment it pays for itself.
Runs on your machine
Sixty seconds to an honest verdict.
# judge one strategy — be honest about how many you tried $ overfitguard validate returns.csv --trials 500 VERDICT: FAILS_OUT_OF_SAMPLE in-sample 2.29 | out-of-sample -0.68 # judge a whole search at once $ overfitguard screen candidates.csv VERDICT: BEST_IS_SIGNIFICANT best: cfg_007 | reality-check p = 0.002
Local by design
Your edge never leaves your laptop.
No account. No upload. No cloud watching what you test. OverfitGuard is a dependency-light Python library — just numpy and pandas — that runs entirely on your machine.
- ✓Private. Your returns stay on your computer, always.
- ✓Framework-agnostic. It never sees your data source, broker, or secret sauce — just a column of returns.
- ✓Open source, MIT. Read every line of the judgment it passes.