In Bittensor, there are two Subnets strongly related to trading: Subnet 8 Propriety Trading Network and Subnet 28 Foundry S&P 500 Oracle. Currently, the proportion of TAO Emissions for the former is approximately 3.82%, while for the latter it is around 1.79%. So, are their current outputs and incentives aligned, and what areas can be optimized in the future?
Subnet 8 Propriety Trading Network（PTN）
Emission：3.82%（2024–07–15）
Github：https://github.com/taoshidev/proprietary-trading-network/tree/main
Subnet 8 provides a simulated trading system where the tradable assets include forex, crypto assets (currently only BTC and ETH), and indices. Traders can trade these assets according to predefined rules and build portfolios. For specific rules, please refer to the official documentation.
Miners act as Traders, submitting Long/Short trading orders within the network. Validators are responsible for processing these orders, tracking the performance of each Miner’s portfolio in real-time, and ranking Miners based on a scoring system. Only the top 25 Miners who are not penalized receive TAO Emissions incentives.
How Do the Scoring and Penalty Mechanisms Work?
The scoring is calculated based on a weighted combination of the return rate, Omega ratio, and Sortino ratio of a Miner’s portfolio. The proportion of TAO Emissions each Miner receives is determined by their score proportion.
However, even if a Miner has a high composite score, they cannot receive incentives if they are penalized. Miners will be penalized if either of the following situations occurs:
Consistency Penalty: If Miners fail to maintain relatively stable trading performance over a consecutive 30-day period, they will be penalized. Stable trading performance encompasses two aspects:
- Miners must remain active, trading on at least 18 out of the 30 days.
- The net value of Miners’ portfolios must not exhibit excessive short-term volatility (fluctuations exceeding 30 times the standard deviation within 6 hours).
Drawdown Penalty: The range for maximum drawdown is limited to between 0.25% and 5%. If a Miner’s maximum drawdown falls below 0.25% or exceeds 5%, they will be penalized. This drawdown assessment considers the entire trading history, not just the 30-day performance window.
For instance, Miner-5GhRddUNcwWSaaa8o5ipcYr4HLCYMg1WwH3rUWdF6RHgE581 holds the highest return rate ranking but has been penalized due to a maximum drawdown exceeding 16%, resulting in almost no incentive allocation. This clearly shows that the PTN encourages stable and relatively conservative investment strategies.
Firstly, in the scoring mechanism, PTN opts to reference the Omega ratio and Sortino ratio, which focus on tail risk and downside risk, rather than solely considering return rates. Additionally, the concept of diversification is embodied in PTN’s long-term objectives. PTN aims not merely to train a specific trading model but to maintain a highly competitive simulated trading leaderboard, from which several outstanding trading models can be selected. By averaging the portfolios of these top-ranked models according to their rankings, PTN derives a composite portfolio that reduces the risk of reliance on a single model.
Although this simulated system appears highly competitive, where models must ensure a certain level of returns while maintaining low drawdowns, can the winning models genuinely be deployed in large-scale real trading?
Regrettably, due to the design flaws of the simulated system, the winning models may not perform as well in live trading as they do in the simulation.
The design of the trading rules in the simulated system has several unreasonable aspects:
1. Ignoring Market Liquidity and Slippage: All trading orders are fully executed at the quoted prices in the simulated system without any delay or price change, which is not reflective of real market conditions.
2. Ignoring the Possibility of Margin Calls: The simulated system does not account for situations where the margin is insufficient and lacks a forced liquidation mechanism.
3. Exaggerating Capital Utilization: While the maximum leverage for each trading pair is limited, there is no reasonable leverage limit for the overall position of Miners. Additionally, it assumes that all positions can share the margin, which is significantly different from real trading systems.
4. Fixed Borrowing and Holding Costs: In reality, these trading costs vary with market fluctuations. Fixed rates might underestimate these costs, thus exaggerating the investment return rate.
5. Limited Order Types: The simulated system essentially only accepts market orders that are fully executed, and does not support even the most basic order types like Stop-loss or Take-profit, which limits the flexibility of strategies.
6. Overly Restrictive Trading Frequency and Holding Period: The trading frequency is restricted to one order every 10 seconds at the fastest, and the shortest holding period must be more than 15 minutes, which also limits strategy flexibility.
The inherent flaws of the simulated system exacerbate the challenges winning models face when applied to real markets:
- Ignoring the Impact on the Market and Competition Pressure in Live Trading: Whether trades in the simulated system can be executed does not consider if the same real orders would be filled in reality. It also ignores the impact these trades have on the market and the reflexivity of trading.
- Underestimating Tail Risk in Winning Miners: Although the scoring mechanism includes metrics for tail risk and downside risk, the flaws in trading rule design may underestimate actual trading costs and overestimate capital utilization. This can lead to an overestimation of the models’ return rates, rendering these metrics potentially inaccurate.
Is anyone actually using Miners’ strategies for real market trading, and how are they performing?
Despite there being a product in the market that follows these strategies, it remains challenging to draw definitive conclusions about their real-world performance.
Dale is a trading bot that operates based on signals provided by Tarvis, the ninth-ranked Miner in PTN. It has been doing trading on Bybit for 45 days, currently has 168 users following its trades, an AUM exceeding 400,000 USDT, and a total profit nearing 20,000 USDT.
For Bittensor, Dale is a commendable attempt as it represents a real-world application where users benefit from Bittensor’s output. Since its launch, a total of 838 users have followed Dale’s trades. Among them, 217 users have made a profit, 305 users have broken even, and 316 users have incurred losses. The highest-earning user used 130,556 USDT, followed the trades for 33 days, and earned 3,871 USDT, achieving a return rate of 2.96%. Conversely, the user with the highest loss used 135,755 USDT, followed the trades for 7 days, and lost 7,503 USDT, with a return rate of -5.52%.
However, since Tarvis’s strategy includes many forex trades and Dale only replicates Tarvis’s BTC and ETH trades at 2x Tarvis’s position size with a 5x exchange leverage, it can only partially reflect Tarvis’s actual performance.
Moreover, despite its impressive performance based on a return rate of +25.98% and a trading win rate of 72%, it is challenging to evaluate this as a stable and consistently profitable strategy as promoted by PTN, considering the overall active period is only 45 days, and the majority of trading profits were concentrated in the week from June 11 to June 18.
Additionally, it should be clarified that the nearly 20,000 USDT profit represents the total earnings of all users following Dale and the bot itself, and should not be simply interpreted as the revenue of SN 8. Even top Validators might misunderstand this point.
Validators should take a more serious approach in considering how to allocate Weight to Subnets. Revenue generation should not be the sole metric; the current output and future potential should also justify the current incentive proportion. Maintaining 11.83% of Emissions means that 851.76 $TAO are allocated to SN 8 daily, equating to over $250,000 in incentives. Rewarding total trading profits of $20,000 with daily releases of $250,000 clearly doesn’t add up to a good deal.
To put it another way, even with the current 3.82% of Emissions, this means that 275.04 $TAO are allocated to SN 8 daily. With such a significant incentive, SN 8 should also perform better.
Subnet 28 Foundry S&P 500 Oracle
Emission：1.79%（2024–07–15）
Github: https://github.com/foundryservices/snpOracle
SN28 builds a network to forecast the S&P 500 index price. Validators send future timestamps to Miners, who then provide the S&P 500 prices for the next six 5-minute intervals after the timestamp. Validators record these forecasts and score Miners based on how close their predictions are to the actual results.
How Does the Scoring Mechanism Work?
SN28 evaluates Miners using two metrics: Root Mean Square Error (RMSE) and Directional Accuracy, with each metric weighted equally at 50%.
- Root Mean Square Error (RMSE): RMSE is the square root of the average of the squared differences between the predicted values and the actual values. The specific formula for RMSE is:
The smaller the RMSE value, the closer the model’s predictions are to the actual values, indicating higher prediction accuracy of the model.
2. Directional Accuracy: Even if Miners are unable to accurately predict the exact values, they are considered directionally correct as long as the predicted direction of change (up or down) is accurate.
So, how accurate are the predictions made by Miners on SN 28?
Based on the backtest data from the past 30 days, the results are underwhelming. The green line represents the actual S&P 500 trend, while the other lines represent Miners’ predictions. The visual clearly shows that there is a significant gap between the predicted values and the actual values, and the directional accuracy is not always correct.
Worse still, SN 28 hardly qualifies as a competitive incentivized subnet. The differences in incentives among different Miners are minimal, with none of them standing out. Currently, there are 312 Miners in the network, with the top Miner receiving just 0.485% of the incentives, and 234 Miners receiving more than 0.4% each. This indicates that the prediction accuracy among most Miners is quite similar and none can be considered highly accurate.
Given SN 28’s current performance, such results offer little practical value.
After understanding the actual operation of these two Subnets, we can address the initial question:
Considering the current output, are the incentives overestimated?
Both SN 8 and SN 28 should perform better to justify the current incentives.
For SN 8, as a TAO Emissions Top 5 Subnet, relying solely on a simulated trading system with numerous flaws is unlikely to be convincing. These flaws may lead to the winning strategies in simulations not performing well in real-world applications. The simulation system might underestimate trading costs and ignore the impact of trades on the market, making some objective metrics unable to accurately evaluate the Miners’ actual performance. Models that succeed in PTN may not be broadly applicable in live trading.
For SN 28, the non-continuous and inaccurate price predictions are even further from practical application. Due to the lack of mechanisms to stimulate effective competition among Miners, even the predictions from top-ranked Miners are unreliable, making them unsuitable for guiding trading decisions.
Areas for Future Optimization
For SN 8: Beyond fixing the flaws in the simulated system, it is crucial to incorporate the actual performance of models into the scoring metrics. Since there are unavoidable differences between simulated systems and real markets, even minor discrepancies can lead to significant differences between simulated and actual performance. Additionally, considering real-world performance would encourage Miners to develop more products similar to Dale, accelerating the process of Bittensor outputs being widely used by real users.
For SN 28: The immediate priority is to establish a more comprehensive scoring mechanism to encourage effective competition among Miners and improve the accuracy of predictions. Furthermore, it is essential to find practical applications for Miners’ outputs. If predictions are made purely for the sake of prediction, it is unnecessary to waste TAO Emissions on a “lottery game” among Miners.
Reference
- https://github.com/taoshidev/proprietary-trading-network/tree/main
- https://docs.taoshi.io/ptn/miner/overview/
- https://dashboard.taoshi.io/miner/5GhCxfBcA7Ur5iiAS343xwvrYHTUfBjBi4JimiL5LhujRT9t
- https://dashboard.taoshi.io/miner/5G3ys2356ovgUivX3endMP7f37LPEjRkzDAM3Km8CxQnErCw
- https://www.bybit.com/copyTrade/trade-center/detail?leaderMark=TwqtPCVsAiXw/1F21f1byQ==&ref=NNBM3N&inviteUuid=2NDbnUXx+LO/7FrPoz5bKm0zT3hZuoOJVO646IKNUbKB038yNU1VuPD25xgDiFnA&af_xp=custom&pid=copy_trade&is_retargeting=true&c=copy_trade-web_to_app&af_force_deeplink=true
- https://github.com/foundryservices/snpOracle
- https://bittensor.foundrydigital.com/
- https://x.com/fish_datura/status/1806801342645583960?s=46&t=sfxHJI4f3g5nVyB50vFXPw
