In the world of cryptocurrency trading, many people often rely on specific "trading indicators." However, numerous papers point out that most trading strategies claiming consistent profitability through backtesting are not proven effective by the market, but rather are selected survivors. Just like getting a perfect score on last year's college entrance exam a hundred times doesn't guarantee you'll perform as well this year—this is the "overfitting" trap of trading strategies. A more practical question is, if a strategy truly performs so well, why not leverage it yourself instead of selling it or sharing it publicly?
After all, truly effective strategies are often limited by their capacity to absorb large amounts of capital; once the capital becomes too large, the advantage can be swallowed up by one's own trading behavior and market reactions.
Trading indicator developers often only raise funds for the best parts of the metrics.
A paper published by the American Mathematical Society points out backtesting bias. The study found that, within the traditional backtesting framework, some technical strategies can indeed generate significant positive returns in historical data, which is a major reason why technical analysis has long been favored by the market. However, the authors further point out that such results often overlook a crucial issue: data-snooping bias.
When researchers test hundreds or even thousands of trading rules simultaneously, a statistically significant minority of strategies will inevitably emerge, even if the market itself is completely random. Judging the effectiveness of technical analysis solely based on these post-hoc winners is tantamount to mistaking luck for skill.
After correcting the biases, the advantages of the technical strategy were significantly reduced.
To address these issues, the study employed more rigorous statistical testing methods, incorporating biases introduced by multiple testing into the adjustment. The results showed that once corrected, almost all technical strategies that initially appeared to offer significant excess returns lost statistical significance. In other words, technical trading rules struggled to reproduce their historical performance out of the sample, indicating that these strategies failed to truly capture sustainable market structures.
After factoring in transaction costs, the actual returns become even more pessimistic.
The study further considers transaction costs. Since technical trading strategies typically involve high turnover rates, when fees, slippage, and market impact costs are factored in, even the few strategies that still maintain a slight positive return often turn negative. The authors point out that this result has high practical significance for real-world trading because most publicly available backtesting results tend to underestimate the friction costs of the actual trading environment.
The study's findings do not completely negate the value of technical analysis, but rather point out that its role is more suited as a risk management, trend identification, or behavioral aid tool, rather than a sole source of profit. In today's highly competitive and rapidly evolving markets, relying solely on historical price and volume signals is no longer sufficient to create a sustainable trading advantage.
Backtesting errors of trading indicators: Using past exam questions to claim excellent test performance.
A paper titled " The Probability of Backtest Overfitting" points out that the seemingly perfect backtest performance you see is highly likely a product of data overfitting. In quantitative finance, backtesting is a standard tool for evaluating the risk and return of a strategy. However, with increased computing power, researchers can now easily test billions of strategy combinations on the same set of historical data.
The authors of the paper point out that this is like saying, "If you interrogate the data long enough, it will eventually confess." When researchers keep adjusting parameters (such as the length of the moving average, entry thresholds, etc.) until the performance looks perfect, they are often just fitting past market noise rather than capturing future signals. Just like you do many past exam questions for the university entrance exam until you see that you can get 100 points on last year's questions, it doesn't mean you will perform the same way on this year's exam, because they are different things.
To address this issue, the research team proposed a key metric: the Probability of Backtest Overfitting (PBO). PBO calculates the probability that "the best-performing strategy in historical backtesting will underperform in the future." A high PBO indicates that the strategy deliberately selects "optimal parameters," while a low PBO indicates that the strategy is robust.
The paper experiments with a strategy boasting a Sharpe ratio of 1.27, which would be highly attractive to most investors. However, testing revealed that this strategy had a PBO (Profit-to-Sales) ratio as high as 55%. Although all in-sample backtests showed positive returns, out-of-sample testing resulted in losses in 53% of cases. This demonstrates that even strategies with high Sharpe ratios can be entirely products of overfitting.
Empirical Study of the Indian Stock Market: RSI and MACD Fail to Consistently Outperform the Market
Having discussed backtesting and statistical issues, let's look at actual research. An 18-year empirical study of the Indian stock market shows that, overall, the technical analysis tools widely used by investors are insufficient to help traders consistently generate excess returns. Even when they show a temporary advantage during some bear markets, their risk-adjusted performance is still insufficient to prove that technical analysis has long-term profitability.
This study, published in the Colombo Business Journal by S. Muruganandan, a scholar at Sri Dharmasthala Manjunatheshwara College in India, uses the Bombay Stock Exchange (BSE) Sensex index as its research subject. It collects historical data from February 2000 to May 2018, spanning multiple bull markets, bear markets, and consolidation periods, to examine the actual profitability of two of the most common technical indicators: the Relative Strength Index (RSI) and the Moving Average Convergence Divergence (MACD).
RSI underperformed across the board, failing to establish a stable advantage in any market cycle.
The study results show that, throughout the overall sample period, the RSI trading strategy, whether using buy or sell signals, did not significantly outperform the unconditional average return of "not using any strategy," and even showed inefficiency before deducting transaction costs.
Further analysis of market cycles reveals that while the RSI frequently issues sell signals during most bull markets, it fails to effectively capture trend continuation. During bear markets or consolidation periods, although buy signals increase, entry is often premature, resulting in continued poor returns. Research indicates that the structural characteristics of the RSI make it prone to contrarian trading in trending markets, thus becoming a drag on performance. From a risk-adjusted perspective, the Sharpe ratio of RSI strategies is mostly negative, indicating that the risk taken has not yielded corresponding returns.
MACD only briefly outperforms sell signals in bear markets.
In comparison, MACD performs slightly better, but it is still not considered stable and reliable. Research has found that MACD buy signals have not significantly outperformed the market in average returns across all market cycles; however, sell signals have indeed shown statistically significant positive returns during most bear markets, outperforming the unconditional average return.
This means that during market downturns, MACD is better able to help traders avoid some of the losses or profit through short-selling strategies. However, when the research further incorporates risk measurement, it finds that even when sell signals are profitable in a bear market, the Sharpe ratio remains low, indicating that the returns are insufficient to compensate for the volatility risk inherent in the strategy itself. In other words, MACD is useful in specific situations, but it is still significantly different from being a reliable long-term profit-making tool.
The study concludes that, under the weak-form efficiency hypothesis, the Indian stock market effectively reflects historical price information, making it difficult to achieve exceptional returns in the long run by relying solely on technical indicators. Even in emerging markets with relatively incomplete information, the advantages of technical analysis are gradually being eroded by the market over time. The authors specifically emphasize that if transaction fees, slippage, and capital costs from actual trading are further factored in, the actual performance of technical analysis strategies may deteriorate even further.
If the strategy in this article is truly effective, why not profit from it yourself? Three papers reveal the brutal truth behind selling metrics . This article first appeared on ABMedia, a ABMedia .
