The Backtest That Lied

How I built, broke, and rejected an A-share machine-learning strategy — and what I learned by killing my own best result.

Student Researcher · July 2026 · experiments · audited metrics

Project Overview

The Question

Can 20 price-volume factors combined with gradient-boosted trees predict next-month A-share returns well enough to build a profitable trading strategy?

The Protocol

3-fold walk-forward validation, 20-seed ensemble, cross-sectional rank normalization, point-in-time universe of 4,073 stocks, next-open execution, transaction-cost scenarios.

The Discovery

In the lowest-liquidity quintile, genuine stock-selection ability exists. But the signal does not generalize to tradable universes. Statistical predictability and economic investability are different things.

The Decision

After 200+ model fits across 6 experimental phases, the OHLCV-only hypothesis was formally terminated. The research branch was closed — not because the model predicted nothing, but because its predictions could not produce scalable positive returns.

The Three Collapses

Loading evidence…

The Signal Was Real. The Strategy Was Not.

Cross-sectional predictability
Time-aware significance
Within-Q1 stock selection
Positive liquid-universe return
Acceptable drawdown
Scalable capacity

The frozen baseline retained a mean out-of-sample IC of with strong time-aware inference (Newey-West t > 9, bootstrap CI excludes zero). The final rejection was not "the model predicted nothing." It was that statistical ranking did not survive the transition to executable, cost-aware, liquidity-filtered portfolios. OHLCV-only development was terminated as a scalable core strategy.

Evidence Exhibits

Loading evidence…

Additional Evidence (4 exhibits)

Loading…

Interactive: Leak Switches

Each switch represents a data or methodological flaw I discovered. Toggle to see how apparent performance changes. Only historically tested combinations are shown.

"I began this project trying to predict stock returns.
I finished it learning how easily a researcher can predict the past."

The hardest skill in research is not finding patterns.
It is killing your own ideas when the evidence says no.