How to Scientifically Evaluate a Quantitative System?

Although our product is significantly different from traditional quantitative strategies, in essence, we are still a quantitative strategy, and thus, the metrics used to evaluate traditional quantitative strategies are applicable. In fact, whenever you encounter any investment opportunity, you can use this framework to evaluate it yourself, as it is the cornerstone of strategy evaluation in the financial field for over a hundred years.

There are 7 metrics to evaluate a quantitative system, divided into three categories:

1 Profitability

• Annualized Return: Measures the performance of the strategy within a year, reflecting its profitability. Most funds in the market use annualized return to compare with each other, so this metric is straightforward, and of course, the higher, the better.
• Sharpe Ratio: The return adjusted for risk, showing the return per unit of risk. It is an important metric to measure the strength of a strategy, not just luck.
• Number of Trades: The trading frequency of the strategy, reflecting the activity level and market participation of the strategy.

2 Risk Control

• Maximum Drawdown (MDD): Represents the maximum potential loss from peak to trough, crucial for understanding downside risk. An excellent strategy needs to be resilient, not collapsing even in the worst market conditions. Generally, controlling it within 10% is considered an excellent strategy.
• Expected Shortfall (Tail Risk): Quantifies the expected loss in tail events, providing insights into extreme risks. It measures the maximum loss a strategy might face when a black swan event occurs. The lower this metric, the better, and it should not exceed 20%.

3 Scalability

• Strategy Capacity: Determines the amount of capital the strategy can manage without degrading performance, crucial for assessing scalability. This metric is often overlooked, but it is very important. If a strategy’s capacity is small, it may not be able to maintain the promised return rate when the capital scale exceeds a certain amount.
• Market Adaptability: Whether the strategy’s performance is stable in different market environments (such as bull markets, bear markets, and volatile markets). An excellent strategy should be able to maintain consistency under various market conditions.

In-depth Analysis of Key Metrics

Below, we will delve into these key metrics:

Annualized Return

Annualized return is the most intuitive metric to measure the profitability of a strategy. It reflects the performance of the strategy within a year, and of course, the higher, the better. But it should be noted that annualized return cannot make any judgment alone and must be evaluated in combination with other metrics.

Sharpe Ratio

The Sharpe ratio is an important metric to measure the risk-adjusted return of a strategy. Its calculation formula is:

Sharpe Ratio = (Average Return - Risk-Free Rate) / Standard Deviation of Return

It has two key points:

• Average Return: Reflects the profit level of each trade, and of course, the higher, the better.
• Standard Deviation of Return: Reflects the volatility of returns. The larger the standard deviation, the greater the volatility of returns, and the higher the risk.

The significance of the Sharpe ratio lies in its ability to eliminate strategies that achieve high returns due to luck. Generally:
A Sharpe ratio over 1 is good; over 2 is considered excellent; over 3 is outstanding.

Number of Trades

It should be noted that a high Sharpe ratio does not necessarily mean the strategy is excellent; there must also be a certain number of trades and market adaptability.
Generally, at least 50 independent trades are needed, and these trades should be distributed across different market conditions (such as bull markets, bear markets, and volatile markets) to have statistical significance.

Maximum Drawdown

Maximum drawdown is an important metric to measure the risk tolerance of a strategy. It reflects the maximum loss a strategy might face during the drawdown process. An excellent strategy needs to maintain resilience in the cyclical fluctuations of the market, and the maximum drawdown should be controlled within 10%. A drawdown exceeding 20% may lead to investor psychological collapse, triggering a wave of redemptions and causing the fund to collapse.

Tail Risk

Tail risk measures the maximum loss a strategy might face when a black swan event occurs. It is a quantitative assessment of extreme risks. The lower this metric, the better, and it should not exceed 20%. Note the difference between it and maximum drawdown; it refers to the strategy’s performance after the worst-case scenario in a power-law distribution.

Strategy Capacity

Strategy capacity refers to the maximum amount of capital a strategy can manage without degrading performance. This metric is very important but often overlooked. If a strategy’s capacity is small, it may not be able to maintain the promised return rate when the capital scale exceeds a certain amount. Therefore, investors need to understand the source of the strategy’s profits: does it rely on arbitrage in a few opportunities, or can it participate in large asset transactions? You certainly do not want to invest in a ‘shrimp’ strategy but hope to invest in a ‘great white shark’ strategy.

Summary

If you encounter someone showing off their trading performance on social media again, you can use the framework in this article to evaluate their strategy. An excellent quantitative strategy must simultaneously satisfy the following 7 principles:

• High annualized return;
• High Sharpe ratio;
• Sufficient number of trades;
• Small maximum drawdown;
• Low tail risk;
• Large strategy capacity;
• Strong market adaptability.

If the other party does not provide complete information, then their showing off is meaningless. Remember, all investment traps are because they do not simultaneously satisfy the above 7 principles.