# Avoiding overfitting > A fitted backtest looks like a found one. Six rules, three warning bands, and how Orbital's trial count, deflated Sharpe and forward testing show how much of a result is luck. Source: https://orbitalaip.com/docs/backtesting/avoiding-overfitting > [!IMPORTANT] Disclaimer > Backtest results are hypothetical and do not predict future performance. Nothing here is investment advice: Orbital computes the rules you supplied and recommends nothing. Orbital is not a registered investment adviser or broker-dealer. Costs are as modelled by the chosen cost profile and may differ from live trading. ## Why a backtest flatters A backtest is a search. Each run tries one configuration. The more you look, the better the best result looks, whether or not the idea works. Orbital counts that search and discounts the [Sharpe ratio](https://orbitalaip.com/docs/glossary#sharpe-ratio) by it. Six rules keep the search small and the result honest. ## Six rules ### 1. Start with a purpose Write the idea down before the first run: what should happen, why, and what would prove it wrong. A rule found by sweeping numbers is one you can only fit. When you save a version, or a run creates one, **What changed?** takes an optional note; use it to say what the change was for. ### 2. Be forward-looking A decision may use only what was known when it was made. The engine decides on closed bars and fills at the next bar's open by default, so it does not peek; see [One bar of delay](https://orbitalaip.com/docs/backtesting/when-trades-happen#one-bar-of-delay). Hindsight enters through what you give it: a list of today's winners, signal rows built with later knowledge, or a **Universe preset**. A preset is a curated list of current index members. Choose the [universe](https://orbitalaip.com/docs/glossary#universe) as you would have chosen it on the period's first day. ### 3. Test several periods Another window adds a run but not a trial for a **Rules** or **Program** strategy, and a new configuration for a **Signals** strategy, whose rows are built for their period. 1. **Pick the window** On the strategy's **Runs** view, drag the window over the data. 2. **Run it** Click **Run this window…**, check the settings and click **Run backtest**. On the Report, **Market events** shows the strategy's return through named stress periods beside the benchmark's. **Rolling statistics** shows whether its Sharpe ratio holds through time. The **Drawdown** table lists the deepest falls behind the [max drawdown](https://orbitalaip.com/docs/glossary#max-drawdown) and when each recovered. ### 4. Mind trade frequency The [win rate](https://orbitalaip.com/docs/glossary#win-rate) of a few trades can be a lucky handful. The Strategies list marks fewer than 30 closed trades as **too few** under **Profitable trades**. Many trades carry the opposite risk: costs decide the result. Check **Trades per day** and **Turnover** in the Report. Returns are net of the commission and slippage the run charges, but the bid-ask spread and market impact are [not modelled](https://orbitalaip.com/docs/backtesting/backtest-vs-forward-test-vs-paper#what-is-not-modelled). ### 5. Change parameters sparingly Every new combination of values is a new configuration and one more trial. Prefer round, defensible numbers, and a setting whose neighbours also work. If a 14-day lookback wins while 12 and 16 lose, the 14 is probably luck. The Runs table's **Setting** column shows the parameter values each run was given, so a sweep is on record. ### 6. Use a reality test Judge the configuration on data it never saw: a later period you held back, or a forward test. On the Report, **Resampling** redraws the closed trades 1,000 times and shows the **Median resample** and the **Worst 1 in 20**. A result that holds on a held-back period, in a forward test and under resampling is worth paper trading. See [Backtest, forward test and paper](https://orbitalaip.com/docs/backtesting/backtest-vs-forward-test-vs-paper). ## How much searching is too much These bands are rough rules of thumb, not Orbital measurements. Orbital records the first row; keep the other two yourself. | Measure | Low risk | Take care | High risk | | --- | --- | --- | --- | | Backtests per strategy | Under 30 | 30 to 70 | Over 70 | | Parameters in the strategy | Under 10 | 10 to 20 | Over 20 | | Hours spent tuning | Under 8 | 8 to 16 | Over 16 | Backtests are every run across the strategy's configurations; parameters are the values you could set differently. Past the middle band, stop tuning and test on data the strategy has not seen. ## What Orbital shows ### The trial count The [trial count](https://orbitalaip.com/docs/glossary#trial-count) counts the distinct configurations of a strategy that have run, whoever launched them, and the Report's **Run properties** states it once there are two or more. See [Versions and configurations](https://orbitalaip.com/docs/backtesting/versions#versions-and-configurations). ### The deflated Sharpe The [deflated Sharpe ratio](https://orbitalaip.com/docs/glossary#deflated-sharpe-ratio) discounts the raw Sharpe for the trial count. It needs at least two trial Sharpes that differ, and a negative figure means the search alone explains the result. The Report's **Probabilistic Sharpe ratio** is a different figure: the chance the true Sharpe is above zero. The deflated figure never appears without the raw Sharpe and the count. In the Strategies list's **List** view, open **Columns** and pick **Full**; on a narrow window the list keeps the **Simple** columns. The **Evidence** column shows the deflated figure over the raw one and the count. Hover it for the whole sentence, such as "Sharpe 1.62 raw · 1.08 after deflating for 47 configurations tried." A strategy whose runs all predate configuration tracking reads **raw · search not tracked**. Chat and [get_backtest](https://orbitalaip.com/docs/claude-and-chatgpt/tools/get_backtest) return the same three figures. The **Sharpe** tile, the statistics tables, the **Runs** table and **Compare** show the raw Sharpe only. ## Hold out with a forward test A [forward test](https://orbitalaip.com/docs/glossary#forward-test) is the cleanest holdout, since everything after its start is data the strategy never saw. Finish the search, start one from the run you would trade, and judge the forward stretch on its own after enough sessions and trades; a week says little. See [Start a forward test](https://orbitalaip.com/docs/backtesting/forward-testing#start-a-forward-test).