Produced by: CGV Research
Author: Cynic, Shigeru
Leveraging the power of algorithms, computational power, and data, the advancement of AI technology is redefining the boundaries of data processing and intelligent decision-making. Meanwhile, DePIN represents the paradigm shift from centralized infrastructure to decentralized, blockchain-based networks.
As the world accelerates its digital transformation, AI and DePIN (Decentralized Physical Infrastructure) have become fundamental technologies driving transformation across all industries. The integration of AI and DePIN not only promotes rapid technological iteration and widespread application but will also initiate more secure, transparent, and efficient service models, bringing profound changes to the global economy.
DePIN: Decentralized Real-World Implementation, A Pillar of the Digital Economy
DePIN stands for Decentralized Physical Infrastructure. In a narrow sense, DePIN mainly refers to distributed networks of traditional physical infrastructures supported by distributed ledger technology, such as power grids, communication networks, and positioning networks. Broadly speaking, any distributed network supported by physical devices can be considered DePIN, including storage and computing networks.
from: Messari
If Crypto has brought decentralized transformation at the financial level, then DePIN is the decentralized solution in the real economy. It can be said that PoW mining machines are a type of DePIN. From day one, DePIN has been a core pillar of Web3.
The Three Pillars of AI — Algorithms, Computational Power, and Data, Two of Which Are contributed by DePIN
The development of artificial intelligence is generally considered to depend on three key elements: algorithms, computational power, and data. Algorithms refer to the mathematical models and program logic that drive AI systems, computational power refers t-o the computing resources required to execute these algorithms, and data is the foundation for training and optimizing AI models.
Which of the three elements is the most important? Before the appearance of ChatGPT, people generally thought it was algorithms, otherwise academic conferences and journal papers wouldn’t be filled with one algorithmic tweak after another. But after the debut of ChatGPT and its supporting large language models (LLM), people began to realize the importance of the latter two. Massive computational power is a prerequisite for the birth of models, and the quality and diversity of data are crucial for building robust and efficient AI systems, making the demand for algorithmic refinement less critical than before.
In the era of large models, the shift from fine-tuning to leveraging substantial resources has increased the demand for computational power and data, and DePIN can provide exactly that. Token incentives leverage the long-tail market, and the vast amount of consumer-grade computational power and storage will become the best nourishment for large models.
Decentralization of AI Is Not an Option, But a Necessity
Of course, some may ask, with computational power and data available in AWS data centers, which offer stability and a better user experience, why choose DePIN over centralized services?
This viewpoint has its reasons, as currently, almost all large models are directly or indirectly developed by large internet companies — ChatGPT is backed by Microsoft, Gemini by Google, and almost every large internet company in China has its own large model. Why? Because only large internet companies have enough quality data and financially supported computational power. But this is not right; people no longer want to be manipulated by internet giants.
On one hand, centralized AI carries data privacy and security risks, and may be subject to censorship and control; on the other hand, AI produced by internet giants further strengthens dependence, leading to market centralization and raising barriers to innovation.
from: https://www.gensyn.ai/Humanity should not need a Martin Luther of the AI era; people should have the right to speak directly with the divine.
A Business Perspective on DePIN: Cost Reduction and Efficiency Increase are Key
Even if we set aside the ideological debate between decentralization and centralization, from a business standpoint, there are compelling reasons to use DePIN in AI.
Firstly, it is important to recognize that despite the fact that internet giants possess a massive amount of high-end graphics card resources, the combination of consumer-grade graphics cards scattered among the public can also form a considerable computational power network, which is the long-tail effect of computing power. These consumer-grade graphics cards have actually a very high idle rate. As long as the incentives provided by DePIN exceed the cost of electricity, users will be motivated to contribute computational power to the network. In addition, since all the physical infrastructure is managed by the users themselves, the DePIN network does not have to bear the operational costs that centralized providers inevitably incur and can focus solely on the protocol design itself.
Regarding data, the DePIN network can unlock the availability of potential data and reduce transmission costs through edge computing and other means. Moreover, most distributed storage networks have automatic deduplication functions, which reduce the workload of cleaning AI training data.
Finally, the crypto-economic models introduced by DePIN enhance the fault tolerance of the system, with the hope of achieving a win-win-win situation for providers, consumers, and the platform.
from: UCLA
In case you doubt, the latest research from UCLA indicates that decentralized computing has achieved 2.75 times the performance of traditional GPU clusters at the same cost. To be specific, it’s 1.22 times faster and 4.83 times cheaper.
A Thorny Path: What Challenges Will AIxDePIN Encounter?
We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard.
— — John Fitzgerald Kennedy
Building artificial intelligence models with DePIN’s distributed storage and distributed computation without trust still poses many challenges.
Work Verification
Essentially, computing deep learning models and PoW (Proof of Work) mining are both forms of general computation, fundamentally based on signal changes between logic gates. On a macro level, PoW mining is considered “useless computation,” as it attempts to produce a hash value with a prefix of n zeros through countless random number generations and hash function computations; whereas the computation in deep learning is seen as “useful computation,” which calculates the parameters of each layer through forward and backward propagation, thereby constructing an efficient AI model.
The reality is that “useless computations” like PoW mining use hash functions, which are easy to calculate from the original image to the hashed image, but difficult to reverse, making it easy for anyone to quickly verify the validity of the computation. In contrast, for computations in deep learning models, due to their hierarchical structure, with each layer’s output serving as the input for the next layer, verifying the validity of the computation requires redoing all previous work and cannot be done simply and effectively.
from: AWS
Work verification is extremely crucial; otherwise, the provider of the computation could submit a randomly generated result without performing any actual computation.
One idea is to have different servers perform the same computational tasks and verify the work’s authenticity by repeatedly executing and checking for consistent results. However, most model computations are non-deterministic, meaning that even in the same computational environment, it is impossible to reproduce identical results; only statistical similarity can be achieved. Moreover, repeated computations would lead to rapidly increasing costs, which contradicts the key goal of cost reduction and efficiency improvement in DePIN.
Another idea is the Optimistic mechanism, which initially trusts that the results are from valid computations, while allowing anyone to verify the results. If errors are found, a Fraud Proof can be submitted, the protocol will penalize the fraudster, and reward the whistleblower.
Parallelization
As previously mentioned, DePIN primarily leverages the long tail of the consumer-grade computing power market, which implies that the computing power provided by a single device is limited. Training large AI models on a single device would take a very long time, necessitating the use of parallelization to shorten the required training time.
The main challenge in parallelizing deep learning training is the dependency between tasks, which makes parallelization difficult to achieve.
Currently, parallelization in deep learning training is mainly divided into data parallelism and model parallelism.
Data parallelism involves distributing the data across multiple machines, with each machine holding a complete set of model parameters and training using its local data, followed by aggregating the parameters from all machines. Data parallelism is effective with large amounts of data but requires synchronous communication for parameter aggregation.
Model parallelism is used when the model is too large to fit on a single machine, allowing the model to be split across multiple machines, with each machine holding a part of the model’s parameters. Communication between different machines is required during forward and backward propagation. Model parallelism has advantages when the model is large but incurs significant communication overhead during propagation.
For gradient information between different layers, there can be either synchronous or asynchronous updates. Synchronous updates are straightforward but increase waiting time; asynchronous update algorithms have shorter wait times but can introduce stability issues.
from: Stanford University, Parallel and Distributed Deep Learning
Privacy
A global movement is underway to protect personal privacy, with governments around the world strengthening the protection of individual data privacy. Although AI heavily uses public datasets, what really distinguishes different AI models is the proprietary user data of various companies.
How can we reap the benefits of proprietary data during the training process without exposing privacy? How can we ensure that the parameters of the AI models constructed are not leaked?
These are two aspects of privacy: data privacy and model privacy. Data privacy protects the users, while model privacy protects the organizations building the models. In the current climate, data privacy is much more important than model privacy.
Various solutions are being explored to address the issue of privacy. Federated learning protects data privacy by training at the source of the data, keeping the data local and only transmitting model parameters; while zero-knowledge proofs may emerge as a promising new solution.
Case Study: What are some high-quality projects in the market?
Gensyn
Gensyn is a distributed computing network for training AI models. The network uses a layer-1 blockchain based on Polkadot to verify whether deep learning tasks have been executed correctly and triggers payment through commands. Founded in 2020, Gensyn disclosed a $43 million Series A funding round in June 2023, led by a16z.
Gensyn uses metadata from a gradient-based optimization process to build certificates of the work performed, and this work is uniformly executed by a multi-granularity, graph-based precision protocol and cross-evaluators to allow re-running of verification work and consistency comparison, with the validity of the calculations ultimately confirmed by the chain itself, ensuring computational effectiveness. To further enhance the reliability of work verification, Gensyn introduces staking to create incentives.
There are four types of participants in the system: submitters, solvers, verifiers, and challengers.
Submitters are the end users of the system, providing tasks to be computed and paying for the completed work units.
Solvers are the main workers of the system, executing model training and generating proofs for verification.
Verifiers are key to linking the non-deterministic training process with deterministic linear computation, replicating part of the solver’s proof and comparing the distance to an expected threshold.
Challengers are the last line of defense, checking the verifiers’ work and issuing challenges, receiving rewards when the challenges are successful.
Solvers need to stake, and challengers check the solvers’ work. If any foul play is discovered, they issue a challenge, and if the challenge is successful, the staked tokens of the solver are confiscated, and the challenger receives a reward.
According to Gensyn’s predictions, this approach is expected to reduce training costs to 1/5 of those of centralized providers.
from: Gensyn
FedML
FedML is a decentralized collaborative machine learning platform for decentralized and collaborative AI anywhere and at any scale. More specifically, FedML provides an MLOps ecosystem for training, deploying, monitoring, and continuously improving machine learning models, while collaborating on a combination of data, models, and computational resources in a privacy-preserving manner. Established in 2022, FedML disclosed a $6 million seed round in March 2023.
FedML consists of two key components: FedML-API and FedML-core, representing the high-level API and the low-level API, respectively.
FedML-core includes two independent modules: distributed communication and model training. The communication module is responsible for underlying communication between different workers/clients and is based on MPI; the model training module is based on PyTorch.
FedML-API is built on top of FedML-core. With FedML-core, new distributed algorithms can be easily implemented through a client-oriented programming interface.
The FedML team’s latest work demonstrates that using FedML Nexus AI for AI model inference on consumer-grade GPU RTX 4090 is 20 times cheaper and 1.88 times faster than on A100.
from: FedML
Future Outlook: The Democratization of AI with DePIN
One day, as AI evolves into AGI, computational power will become the de facto universal currency, and DePIN will have made this process happen sooner.
The fusion of AI and DePIN has opened a new technological growth point, offering tremendous opportunities for the development of artificial intelligence. DePIN provides AI with a vast amount of distributed computational power and data, which helps in training larger-scale models to achieve stronger intelligence. At the same time, DePIN also makes AI develop in a more open, secure, and reliable direction, reducing dependence on single centralized infrastructure.
Looking to the future, AI and DePIN will continue to develop in synergy. Distributed networks will provide a strong foundation for training super-large models, which will play an important role in the application of DePIN. While protecting privacy and security, AI will also help optimize DePIN network protocols and algorithms. We look forward to AI and DePIN bringing about a more efficient, fairer, and more trustworthy digital world.
Reference:
----------------------
About Cryptogram Venture (CGV):
CGV (Cryptogram Venture) is a crypto investment institution headquartered in Tokyo, Japan. Since 2017, its fund and predecessor funds have participated in investing in over 200 projects, including the incubation of the licensed Japanese yen stablecoin JPYW. CGV is also a limited partner in several globally renowned crypto funds. Since 2022, CGV has successfully hosted two editions of the Japan Web3 Hackathon (TWSH), supported by Japan's Ministry of Education, Culture, Sports, Science and Technology, Keio University, NTT Docomo, and other institutions and experts. CGV has branches in Hong Kong, Singapore, New York, Toronto, and other locations. Additionally, CGV is a founding member of the Bitcoin Tokyo Club.
About AIFocus:
AIFocus is an entrepreneurial project accelerator based in Hong Kong, co-initiated by CGV (Cryptogram Venture) and Web3 Labs. It is dedicated to in-depth exploration of the AI and Web3 domains, selecting teams with forward-thinking ideas and creativity, and providing support in terms of technical talent, resources, and funding. AIFocus adheres to the service principle of investing first and then accelerating, aiming to unearth outstanding projects and propel them to rapid success.
Disclaimer:
The information and materials introduced in this article are sourced from public channels, and our company does not guarantee their accuracy or completeness. Descriptions or predictions involving future situations are forward-looking statements, and any advice and opinions provided are for reference only and do not constitute investment advice or implications for anyone. The strategies our company may adopt could be the same, opposite, or unrelated to the strategies readers might speculate based on th
Comments