Home › Documentation › Validation & robustness
Analysis 5

Survival Test

It opens from the Portfolio → Survival Test… menu. It puts one or several SPECIFIC strategies through a battery of robustness tests and issues a single verdict: the AQ Survival Score (0–100). It answers the question: is it genuinely good, or is it "beautiful garbage" (splendid in-sample, collapsing the moment you perturb it)? Send strategies to the module by right clicking in the Mass Test or the Portfolio.

6 phases, a weighted average (Walk-Forward dominates)

Each phase produces a 0–100 subscore. The final score is the weighted average of the phases that were evaluated; phases with no data (other markets not loaded, for instance) do not penalize — their weight is shared among the available ones.

Weights
  • → Walk-Forward (OOS)35
  • → Monte Carlo20
  • → Parameters15
  • → Markets10
  • → Costs10
  • → Timeframes10
What each phase measures
  • →Walk-Forward: the strategy applied over successive OOS windows (% profitable + consistency)
  • →Monte Carlo: % of trade shuffles that stay profitable
  • →Parameters: the stability of each period's neighbors
  • →Markets: how much it degrades on a different underlying
  • →Costs: profit under commission ×1.5/×2/×3 + slippage (1/2/3 ticks)
  • →Timeframes: how much it degrades on a different timeframe
≥ 85 ★★★★★
Fit for real combat
≥ 70 ★★★★
Promising
≥ 55 ★★★
Doubtful
≥ 35 ★★
Overfitted
< 35 ★
Beautiful garbage

Six details that make the score mean something

A robustness test is easy to write and hard to make honest: almost every possible oversight pushes the same way, toward giving weak strategies a free pass. These six details are what separate a score that informs from one that reassures for no reason.

The costs really do get stressed

The tick value is looked up by contract, not by the symbol's text: «@NQ Daily» and «@NQ» are the same one. If they did not match, the tick value would come out as zero, only the commission would be stressed, and with the commission at zero the test would score 100% without examining anything. And slippage moves the price rather than being subtracted from the profit, which is what actually shifts your targets. In the hard scenario that is the difference between 95.5% and 88.3%.

It does not cross against itself

The test runs your strategy on the other markets, and its own market does not count: it is profitable there, that is why you saved it. Counting it would be handing yourself a free pass. In the same way, «@NQ», «@NQ_90» and «@NQ_5» are the same NASDAQ seen in three timeframes, not three markets. In a real case, getting that right moves the score from 90.9 to 87.5.

The parameters include the exit

It moves the periods up and down to see whether the strategy depends on one exact number, and the exit ones count just as much as the entry ones. If your exit is «SMA(11) < Close», that 11 gets tested too. This is no quibble: in a real strategy, seven of the ten periods were in the exit — looking only at the entry would have examined three.

The timeframe is inferred from the data

Not from a saved label or the file name, both of which can lie. It matters because useless data there raises no error: the neighboring-timeframe test would be skipped in silence and the strategy would end up validated with five tests instead of six without anything warning you. A failure you cannot see is worse than one you can.

Every market pays its own costs

When crossing, the strategy inherits all four things from the destination market: point value, commission, tick size and slippage. Inheriting only the point value would be a subtle error — an @ES strategy tested on @CL would pay @ES commission and use @ES slippage —, and then the result would not tell you whether the logic holds there, but how similar the two markets' costs are.

Slippage reaches this far

The one you set per contract in the Quote Manager is applied across the whole Test and in the Multi-Timeframe, along with the session-hours filter. And it has a knock-on effect that gets overlooked: the cost stress adds its ticks to whatever the strategy already pays, so if it already paid zero, the stress would be softer than it claims to be.

How much slippage weighs, measured. With slippage at zero —the usual case if you have not touched it— the score does not notice. Where you do set it, it does: across 1,177 strategies on @ES with 1 tick, profit falls by 1.4% on average, with 924 going down and 253 going up. They go up because slippage moves the price, it does not subtract from the profit: sometimes that worse price prevents an entry that would have gone badly.

Review the Quote Manager before trusting the cross. Each symbol's commission, tick and slippage are genuinely used when crossing markets. A symbol with those numbers at zero will give you an over-optimistic cross, and nothing on screen will warn you.

A detail in the same spirit: the Monte Carlo "worst drawdown" is measured from the peak reached, not from the starting point — measured from the origin, a profitable strategy would post zero almost every time. It is informational and does not score, but it measures the same thing as the rest of the program.

Try it yourself

AniQuant can be tried free for 30 days, with every module and no card.

← Previous
Monte Carlo simulation
Next →
Walk-forward analysis
More in Validation & robustness