A strategy makes money for six trades. Is it good? A setup loses four in a row. Is it broken? Almost every bad decision in systematic trading starts by answering one of those questions too early.
Small samples lie, and they lie confidently
Flip a fair coin ten times and you'll get eight heads about 4% of the time. Nothing is wrong with the coin — that's just what ten flips does. Now replace "heads" with "winning trade" and notice that a strategy with no edge whatsoever will produce a 6-2 stretch often enough that you'll see one this month.
The uncomfortable part: that 6-2 run looks exactly like a real edge. It has a story, a chart, and a plausible mechanism. The only thing separating it from noise is sample size, and sample size is invisible in the moment.
Every losing strategy has a winning week. That's what makes them so hard to kill.
Rough numbers
There's no single magic threshold, but some working intuition:
Under ~30 trades: you are reading noise. You can see whether the mechanics work — orders fill, exits fire, nothing crashes — but you cannot judge the edge.
30-100 trades: a signal starts to emerge, but the error bars are still wide enough to drive through. A result here is a hypothesis, not a finding.
100-400+ trades: now you're talking. Not certainty — the market changes underneath you — but enough that a clear result probably isn't luck.
And the smaller the edge, the more trades you need to see it. A big obvious edge shows up fast. A thin one hides in variance for hundreds of trades. Most real edges are thin.
The trap: slicing
Here's where good traders go wrong. You have 200 trades — a decent sample. Then you ask "how does this do on Tuesdays, in the morning, in high volatility?" and suddenly you're looking at 11 trades. You've cut a real sample into a dozen rumours, and one of those slices will look spectacular by pure chance.
That's data mining bias, and it's the reason a strategy that looked perfect in testing falls apart live. Every extra filter you add costs you sample size — the exact currency you need to know if the filter helps.
What to do instead
Write the question down first. Decide what you're testing and what result would change your mind, before you look. A hypothesis formed after seeing the data isn't a test, it's a description.
Change one thing at a time. Two changes at once and a hundred trades later, you know the pair did something. You don't know which one, and you never will.
Prefer deleting to adding. Removing a rule that never had evidence costs you nothing. Adding one on twelve trades costs you the next six months of confusion.
Let losing streaks be boring. A positive-expectancy system will hand you four losses in a row regularly. That's variance, not information. The plan should already have expected it.
The discipline
The instinct to fix a system after a bad day is the single most expensive reflex in trading. It converts noise into permanent code. The counter-discipline is simple to say and hard to do: measure everything, change almost nothing, and only when the record — not the day — asks for it.
See also: overfitting, backtesting basics, and expectancy.