Overfitting, out-of-sample and walk-forward tests explained
How strategies get fitted to the past, and the three tests professionals use to tell a real trading edge from a lucky curve.
Give a computer enough settings to try and it will always find a combination that made money in the past. That is overfitting: the rules have learned the noise of one particular stretch of history instead of a pattern that repeats. The cure is to judge the rules on data that played no part in choosing them.
In-sample and out-of-sample
Split the history. Use the first part, the in-sample period, to design and tune. Then run the finished rules once on the second part, the out-of-sample period, without changing anything. If the edge is real, the second part should look broadly similar. Some loss of performance is normal and expected; a complete collapse is a warning sign.
Walk-forward testing
Walk-forward repeats the split through time: tune on a window, test on the next stretch, slide forward, repeat. Joining all the test stretches together gives a record made only from data the rules had not seen at the time, which is the closest a backtest gets to live trading.
Forward testing
The final check is time itself. Running the strategy in real time, on a demo or small account, produces results that cannot have been fitted to anything. This is why the Arrissa REX 9 AI backtest is extended automatically every day with newly recorded market data, and why every new user starts with a free simulated demo.
Signs a result is overfitted
- Lots of settings with oddly precise values.
- Performance that falls apart when a setting is nudged slightly.
- A great result from one market only, or one short period only.
- An equity curve that looks too smooth for the number of trades.
Start the bot, watch your free $10,000 demo trade, and connect your Exness account when you are ready.
Read next: How many trades do you need? Sample size and start-date tests
Risk warning. Trading leveraged products such as CFDs carries a high risk of losing money. Backtest figures are simulations, not real results, and past performance does not guarantee future results. Read the risk disclosure before you connect an account. Nothing here is financial advice.