Harnessing the power of algorithms, computing power and data, advances in AI technology are redefining the boundaries of data processing and intelligent decision-making. At the same time, DePIN represents a formalized shift from centralized infrastructure to decentralized, blockchain-based networks.
As the world's pace of digital transformation continues to accelerate, AI and DePIN (decentralized physical infrastructure) have become potential foundational technologies that drive change in all walks of life.
DePIN: Decentralization moves from virtuality to reality, the mainstay of the digital economy
DePIN is the abbreviation of Decentralized Physical Infrastructure. In a narrow sense, DePIN mainly refers to the decentralized network of traditional physical infrastructure supported by distributed ledger technology, such as power network, communication network, positioning network, etc. Broadly speaking, all decentralized networks supported by physical devices can be called DePIN, such as storage networks and computing networks.
If Crypto has brought about decentralized changes at the financial level, then DePIN is a decentralized solution in the real economy. It can be said that the PoW mining machine is a kind of DePIN. DePIN has been a core pillar of Web3 from day one.
The three elements of AI—algorithm, computing power, and data, DePIN exclusively possesses the second one
The development of artificial intelligence is generally considered to depend on three key elements: algorithms, computing power and data. Algorithms refer to the mathematical models and program logic that drive AI systems. Computing power refers to the computing resources required to execute these algorithms. Data is the basis for training and optimizing AI models.
Which of the three elements is the most important? Before the emergence of ChatGPT, people usually thought of it as an algorithm, otherwise academic conferences and journal papers would not be filled with algorithm fine-tuning one after another.
But after the debut of ChatGPT and the large language model LLM that supports its intelligence, people began to realize the importance of the latter two. Massive computing power is the prerequisite for the birth of models. Data quality and diversity are crucial to building a robust and efficient AI system. In contrast, the requirements for algorithms are no longer as refined as usual.
In the era of large models, AI has changed from meticulous craftsmanship to vigorous flying bricks. The demand for computing power and data is increasing day by day, and DePIN can provide it. Token incentives drive the market, and massive consumer-grade computing power and storage will provide the best nourishment for large models.
Decentralization of AI is not an option, but a must
Of course, some people will ask, since computing power and data are available in AWS computer rooms, and they are better than DePIN in terms of stability and user experience. Why should we choose DePIN instead of a centralized service?
This statement naturally makes sense. After all, looking at the current situation, almost all large models are developed directly or indirectly by large Internet companies. Behind ChatGPT is Microsoft, behind Gemini is Google, and China’s Internet giants Almost everyone in the factory has a large model. Why? Because only large Internet companies have enough high-quality data and computing power supported by strong financial resources. But this is wrong. People no longer want to be controlled by Internet giants.
On the one hand, centralized AI carries data privacy and security risks and may be subject to censorship and control; on the other hand, AI produced by Internet giants will further strengthen people's dependence, lead to market concentration, and increase barriers to innovation.
Humanity should no longer need a Martin Luther in the AI era. People should have the right to talk directly to God.
DePIN from a business perspective: Cost reduction and efficiency increase are key
Even putting aside the value debate between decentralization and centralization, from a business perspective, using DePIN for AI still has its merits.
First of all, we need to clearly realize that although Internet giants have a large number of high-end graphics card resources in their hands, the combination of consumer-grade graphics cards scattered among the private sector can also form a very considerable computing power network, that is, computing power long tail effect.
The idle rate of this type of consumer-grade graphics card is actually very high. As long as the incentives provided by DePIN can exceed the electricity bill, users will have the incentive to contribute computing power to the network. At the same time, all physical facilities are managed by the users themselves. The DePIN network does not need to bear the unavoidable operating costs of centralized suppliers, and only needs to focus on the protocol design itself.
For data, the DePIN network can release the availability of potential data and reduce transmission costs through edge computing and other methods. At the same time, for most distributed storage networks, it has automatic deduplication function, which reduces the work of AI training data cleaning.
Finally, the Cryptoeconomics brought by DePIN enhances the system's fault tolerance and is expected to achieve a win-win situation for providers, consumers, and platforms.
In case you don't believe it, UCLA's latest research shows that using decentralized computing achieves 2.75 times better performance than traditional GPU clusters at the same cost. Specifically, it is 1.22 times faster and 4.83 times cheaper.
Difficult road ahead: What challenges will AIxDePIN encounter?
We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard. ——John Fitzgerald Kennedy
There are still many challenges in building artificial intelligence models without trust using DePIN's distributed storage and distributed computing.
Work verification
In essence, computing deep learning models and PoW mining are both general calculations, and the lowest layer is the signal changes between gate circuits. From a macro perspective, PoW mining is a "useless calculation", trying to obtain a hash value with n zeros at the beginning through countless random number generation and hash function calculations; while deep learning calculations are "useful calculations". Forward derivation and backward derivation calculate the argument values of each layer in deep learning to build an efficient AI model.
The fact is that "useless calculations" such as PoW mining use hash functions. It is easy to calculate the image from the original image, but it is difficult to calculate the original image from the image, so anyone can easily and quickly verify the validity of the calculation; For the calculation of the deep learning model, due to the hierarchical structure, the output of each layer is used as the input of the next layer. Therefore, verifying the validity of the calculation requires performing all previous work, and cannot be verified simply and effectively.
Work verification is very critical, otherwise the provider of the calculation could not perform the calculation at all and submit a randomly generated result.
One idea is to have different servers perform the same computing tasks, and then verify the effectiveness of the work by repeating the execution and checking whether it is the same. However, the vast majority of model calculations are non-deterministic, and the same results cannot be reproduced even under the exact same computing environment, and can only be similar in a statistical sense. In addition, double counting will lead to a rapid increase in costs, which is inconsistent with DePIN's key goal of reducing costs and increasing efficiency.
Another type of idea is the Optimistic mechanism, which first optimistically believes that the result is effectively calculated, and at the same time allows anyone to check the calculation result. If an error is found, a Fraud Proof can be submitted. The agreement will fine the fraudster and report it. be rewarded.
Parallelization
As mentioned before, DePIN mainly leverages the long-tail consumer computing power market, which means that the computing power that a single device can provide is relatively limited. For large AI models, training on a single device will take a very long time, and parallelization must be used to shorten the training time.
The main difficulty in parallelizing deep learning training lies in the dependency between previous and subsequent tasks, which makes parallelization difficult to achieve.
Currently, the parallelization of deep learning training is mainly divided into data parallelism and model parallelism.
Data parallelism refers to distributing data across multiple machines. Each machine stores all the parameters of a model, uses local data for training, and finally aggregates the parameters of each machine. Data parallelism works well when the amount of data is large, but requires synchronous communication to aggregate arguments.
Model parallelism is when the size of the model is too large to fit on a single machine, the model can be split across multiple machines, and each machine stores a part of the parameters of the model. Forward and backward propagation require communication between different machines. Model parallelism has advantages when the model is large, but the communication overhead during forward and backward propagation is high.
Gradient information between different layers can be divided into synchronous updates and asynchronous updates. Synchronous update is simple and direct, but it will increase the waiting time; the asynchronous update algorithm has a short waiting time, but will introduce stability problems.
privacy
The global trend of protecting personal privacy is rising, and governments around the world are strengthening the protection of personal data privacy security. Although AI makes extensive use of public data sets, what really differentiates different AI models is the proprietary user data of each enterprise.
How to get the benefits of proprietary data during training without exposing privacy? How to ensure that the parameters of the built AI model are not leaked?
These are two aspects of privacy, data privacy and model privacy. Data privacy protects users, while model privacy protects the organization that builds the model. In the current context, data privacy is much more important than model privacy.
A variety of solutions are being attempted to address the issue of privacy. Federated learning ensures data privacy by training at the source of the data, keeping the data locally, and transmitting the model parameters; and zero-knowledge proof may become a rising star.
Case analysis: What high-quality projects are there on the market?
Gensyn
Gensyn is a distributed computing network used to train AI models. The network uses a layer of blockchain based on Polkadot to verify that deep learning tasks have been executed correctly and trigger payments through commands. Founded in 2020, it disclosed a US$43 million Series A round of financing in June 2023, led by a16z.
Gensyn uses the metadata of a gradient-based optimization process to build certificates of the work performed, consistently executed by a multi-granular, graph-based precision protocol and cross-evaluator to allow the validation work to be re-executed and compared for consistency, and ultimately by the chain Confirm it yourself to ensure the validity of the calculation. To further enhance the reliability of work verification, Gensyn introduces staking to build incentives.
There are four types of participants in the system: submitters, solvers, verifiers and reporters.
Submitters are end users of the system who provide tasks to be computed and pay for completed units of work.
The solver is the main worker of the system, performing model training and generating proofs for inspection by the verifier.
The validator is key to bridging the non-deterministic training process with deterministic linear computation, replicating partial solver proofs and comparing distances to expected thresholds.
The whistleblower is the last line of defense, checking the work of the verifier and raising challenges, and is rewarded after the challenge is passed.
The solver needs to make a pledge, and the whistleblower tests the solver's work. If he discovers evildoing, he will challenge it. After the challenge is passed, the tokens pledged by the solver will be fined and the whistleblower will be rewarded.
According to Gensyn's predictions, this solution is expected to reduce training costs to 1/5 of those of centralized providers.
FedML
FedML is a decentralized collaborative machine learning platform for decentralized and collaborative AI, anywhere and at any scale. More specifically, FedML provides an MLOps ecosystem to train, deploy, monitor, and continuously improve machine learning models while collaborating on combined data, models, and computing resources in a privacy-preserving manner. Founded in 2022, FedML disclosed a $6 million seed round in March 2023.
FedML consists of two key components: FedML-API and FedML-core, which represent high-level API and low-level API respectively.
FedML-core includes two independent modules: distributed communication and model training. The communication module is responsible for the underlying communication between different workers/clients and is based on MPI; the model training module is based on PyTorch.
FedML-API is built on FedML-core. With FedML-core, new decentralized algorithms can be easily implemented using a client-oriented programming interface.
The latest work of the FedML team has proven that using FedML Nexus AI to perform AI model inference on the consumer GPU RTX 4090 is 20 times cheaper and 1.88 times faster than A100.
Future Outlook: DePIN brings the democratization of AI
One day, when AI further develops into AGI, computing power will become the de facto universal currency. DePIN makes this process happen in advance.
The integration of AI and DePIN has opened up a new technological growth point and provided huge opportunities for the development of artificial intelligence. DePIN provides AI with massive distributed computing power and data, helping to train larger-scale models and achieve greater intelligence. At the same time, DePIN also enables AI to develop in a more open, secure, and reliable direction, reducing reliance on a single centralized infrastructure.
Looking to the future, AI and DePIN will continue to develop together. The decentralized network will provide a strong foundation for training very large models, and these models will play an important role in the application of DePIN. While protecting privacy and security, AI will also help optimize the DePIN network protocol and algorithm. We look forward to AI and DePIN bringing a more efficient, fairer, and more trustworthy digital world.
