Home › Documentation › Validation & robustness
Analysis 3

Parameter Robustness Test

Available in the Robustness tab of the Performance window. It answers the most important question once you have found a good strategy: does it only work with exactly these parameters, or also with the values around them?

A strategy that only wins with "CCI of 7 and an exit after 10 bars", and loses with 6 or 11, has not found anything: it has found a coincidence of the past. One that wins with almost every nearby value has found real market behaviour, and will survive a future that is not identical to the past.

Robustness tab: map of the CCI-710 system, CCI period by bars until exit, comparing 1995-2008 with 2009-2026
The CCI-710 system on the S&P 500: CCI period (rows) by bars until exit (columns), comparing 1995-2008 with 2009-2026. 288 backtests.

What each cell is

Each cell is a full backtest of the whole strategy, with a different value of the two chosen parameters. The row gives the value of the first, the column the value of the second, and the number is the result of that combination in the chosen metric.

Example. In the picture, the cell in row 7 and column 10 reads 78 in the left panel: with a CCI of 7 and the exit after 10 bars, the system made $78,000 per contract between 1995 and 2008. The right panel reads 56: the same combination made $56,000 between 2009 and 2026. It is not the number of winning trades: it is the profit of all the trades of that combination.
Colour

Green when it wins, red when it loses; the stronger, the further from the neutral point. Both panels share the same scale, so they compare at a glance.

Orange box

Your strategy as it stands. What matters is what surrounds it, not its own number.

Count

Above each panel: how many combinations make money (or beat PF 1, or a 50% win rate). "144 of 144" is a perfect plateau.

Parameters move together

AniQuant detects every parameter of the strategy and links them: the same parameter changes at once in every condition where it appears. That way each cell is a coherent strategy, the one you would actually write with that value.

  • ·Indicator period: in CCI-710 the CCI(7) appears in the long entry, in the short entry and compared with itself one bar earlier. Moving to 9 changes all of them to 9.
  • ·Threshold: −100 (longs) and +100 (shorts) are a single "±100" parameter. Testing 120 gives −120 for longs and +120 for shorts.
  • ·Bars until exit (BarsSinceEntry), maximum bars, take profit and stop loss, when the strategy uses them.

Price series (Close, Close[5]) are not listed: they have no parameter worth sweeping.

Step by step

①Open the Performance window of a strategy: from the AQL Editor (Test button), Mass Testing, the Library or Genesis. The Robustness tab appears when the window has the series and the strategy.
②In Vertical axis, pick the first parameter and its range from / to / step. A range around the current value is suggested; change it as you like.
③In Horizontal axis, the second one. With "none" you get a single column: a sweep of one parameter.
④Optional: tick Compare two periods and choose the cutoff date (see below).
⑤Press Calculate. Each axis takes up to 25 values; a 12 × 12 grid is 144 backtests and usually takes a few seconds.
⑥Switch metric (Profit ($), Profit Factor (%), Profit Factor ($), Win rate) whenever you want: the map is redrawn without recalculating.
⑦Click a cell: it is boxed in white in every panel at once, and its detail for each period appears above (profit, trades, win rate and both Profit Factors). A second click clears it. Hovering over any cell shows the same detail.

Comparing two periods: the acid test

With the cutoff date, each cell is split in two: the trades that started before and those that started after. Two maps appear side by side, with the same colour scale.

Use it to separate the period you designed the strategy with from the period it never saw. If the green plateau on the left stays green on the right, the idea has survived the future. If it fades, it was a coincidence of the design period.

In the picture: CCI-710 was published in 2008. Between 1995 and 2008 all 144 combinations win; between 2009 and 2026, years the system never saw, 139 of 144 still win. The five that lose sit together in one corner (slow CCI with a very fast exit). The idea is solid. Notice too that the strongest zone has shifted slightly, towards CCI 8 to 10: that is why it pays to pick parameters from the centre of a plateau rather than the highest peak.
Robust strategy

The orange box sits inside a wide green plateau. Neighbouring values win too: the strategy does not depend on hitting the exact parameter.

Overfitted strategy

The box is an isolated peak surrounded by red or pale green. Any small change destroys the result: it is luck, not edge.

Choosing the parameters

If the plateau is wide, take a value from its centre: if the market drifts a little, you will still be inside the good zone.

Details worth knowing

  • ·Profit is per contract, with the symbol's point value and commission. Above $10,000 the map shows it in thousands of dollars.
  • ·Profit Factor (%) weighs every trade the same whatever the price level (the one used in rankings); Profit Factor ($) is the one TradeStation or AmiBroker report. See the Summary tab.
  • ·A grey cell with "—" has fewer trades than the required minimum: it is not rated.
  • ·The periods of a comparison may have different lengths (14 years against 18 in the picture): compare how many cells win and where the plateau is, not the absolute figures.
  • ·The map uses the same engine as the backtest: the orange cell matches the result in the Summary tab.

Equity tab: in and out of sample

  • ·The out-of-sample (OOS) stretch is shaded and labelled («IN-SAMPLE» / «OUT-OF-SAMPLE (OOS)»), AQ Genesis strategies included.
  • ·A grey buy-and-hold 1 contract line from the first trade, for comparison. Turn it on with its checkbox in the legend; it is hidden while Position Size is active.
  • ·Underneath, a panel with the drawdown of both curves.

The Summary tab tells apart two Profit Factors: (%), computed with each trade's percentage (the one every ranking uses), and ($), gains ÷ losses in money, the one TradeStation and AmiBroker report. With prices that rise a lot they can differ considerably.

Try it yourself

AniQuant can be tried free for 14 days, with every module and no card. If you have the book, 30.

← Previous
Export strategies to your platform
Next →
Monte Carlo simulation
More in Validation & robustness