I read a post on an overseas quant community that resonated so much I had to write it down. The title went roughly like this:
“The single biggest gap between my backtest and my real returns came down to one thing.”
After reading it, I got curious — “wait, is my bot okay?” — and went and actually checked.
The verdict: thankfully my bot held up, and as a bonus I walked away with a solid new diagnostic tool.
1. What was the post about?
The author said they’d been wrestling with it for a year. A strategy that clearly made money when you ran it through historical data (this is called a backtest) — but the moment they put it into live trading, returns kept getting shaved by 20–50%.
The culprit turned out to be surprisingly simple.
An invisible cost called “spread.”
Picture a currency exchange counter. It’s the same dollar, but the rate when you buy and the rate when you sell are different.
That gap is the counter’s cut.
Stocks, crypto — same idea. There are always two prices in the market.
- To buy, you have to pay a slightly higher price (what the seller is asking)
- To sell, you have to accept a slightly lower price (what the buyer is offering)
The gap between those two is the spread. The trouble is, most backtesting software just assumes you bought and sold at the midpoint between them. The problem? That midpoint isn’t a price that actually exists. So a cost the backtest never paid gets quietly tacked onto every single live trade.
What makes this scary is that the more frequently you trade, and the more illiquid the asset (= fewer people lined up to buy), the wider that gap gets.
The gap is very narrow on big blue-chip stocks, but on obscure small-cap coins, it’s much wider. If your strategy lives on small edges, a single gap like this can swallow the whole thing.
2. So, is my bot okay? — Checking it myself
My crypto bot runs five sub-strategies in parallel.
Most of them only touch the deeply liquid stuff like Bitcoin or large-cap coins, where this gap is basically a non-issue. Just one — the mean-reversion layer that bets on “coins that have drifted too far from trend will snap back” — actually touches 60 thinly-traded mid- and small-cap coins. So if anything were going to get caught in the spread trap, it would be this guy.
So I pulled out that one strategy and deliberately inflated the trading cost to rerun it.
“What if, in reality, this gap is much wider than I’ve been assuming?”
- I cranked the original assumed cost up to 4× (basically applying the worst-case of the most illiquid coin to everything)
- And checked whether the strategy could still survive
3. The result
Even at 4× the cost, it still made money. To completely break the strategy, I had to push the cost up to nearly 6×. Which means: even under the worst-case assumption of the “invisible cost” I’d been worried about, this strategy holds.
Pass.
(+)
And then came a funny twist.
“So if I just drop the totally illiquid junk coins, wouldn’t it be even safer?”
I tried it. The result: returns actually dropped, and the risk (drawdown) got worse.
Turns out those thinly-traded mid- and small-caps were actually both a source of return and a diversifier of risk. So the right answer wasn’t “they look risky, cut them out,” it was “keep them in, but assume costs conservatively and verify it still holds.”
4. What I took from it
Honestly, the real lesson here isn’t the result — it’s the habit.
- The prettier the numbers look, the more I should ask “is there a hidden cost missing from this return?” first. When a backtest looks too good, it usually means one or two real-world costs got quietly skipped.
- Before actually deploying a new strategy, always run a “does it still survive at several-times the cost?” check. I built that diagnostic this time around, so from now on, any strategy that touches illiquid assets has to pass through this gate. No exceptions.
Running a business feels the same way. Look at revenue alone and it seems like you’re making money — but once you fold in all the easy-to-miss costs like card processing fees, shipping, returns, there’s often not much left. In the end, the real question is “does this still come out ahead once every real cost is in?” Whether it’s quant trading or running a shop.
Learned one more thing today. The habit of suspecting the invisible costs first. Honestly, that might be the most expensive lesson there is.
댓글