Empowering AI Training with Akash Network
Overclock Labs, the mastermind behind Akash Network, partnered with ThumperAI, a generative AI startup, to embark on a groundbreaking model training project utilizing Akash Network’s GPUs. This joint effort, which unfolded in late 2023, aimed to revolutionize decentralized cloud computing by training a foundational model, showcasing a significant advancement in the integration of AI and cloud computing.
Utilizing Akash Network for AI Training
The collaboration targeted the obstacles linked to generative AI foundation model training, such as costly expenses, strict hardware prerequisites, and intricate software requirements. This initiative established an ambitious standard for decentralized computing platforms to meet and surpass the demands of AI startups.
- The initiative addressed challenges in generative AI model training
- It aimed to surpass current demands in AI and cloud computing integration
ThumperAI’s objective was to enhance the reliability of its Lora Trainer service, aiding AI developers in refining foundation models via LoRA techniques. On the flip side, Akash Network sought to showcase the practicality and efficiency of AI model training on a decentralized cloud platform.
- ThumperAI aimed to boost the authenticity of Lora Trainer service
- Akash Network aimed to exhibit the effectiveness of AI model training in a decentralized setting
This initiative was not solely about validating a concept but also about enticing a broader developer community, creating a demand for GPU providers, and solidifying Akash Network’s position as a front-runner in open-source contributions and the expansion of AI use cases.
Challenges and Outcomes
The project involved crucial decision-making, notably in selecting the model category and base model for training. Despite encountering challenges with the training dataset’s quality and diversity, the chosen approach utilized a Creative Commons-licensed dataset for training, emphasizing copyright adherence and data variety, pivoting to a Pixart-Alpha-inspired architecture to overcome initial training hurdles.
- Critical decisions were made in model selection and training approach
- The project faced challenges with dataset quality and diversity
Despite these obstacles, the project’s outcomes were promising as it showcased the viability of training foundational models on a decentralized network like Akash, marking a significant milestone in the convergence of AI and blockchain technology.
“The Thumper team will look into migrating Lora Trainer to run on Akash Network as they look to scale that service, so as to take advantage of the lower costs and variety of GPUs available on the network. One of the features needed before they can do this is the ability to request a max SHM size through the Akash SDL. That feature is about to be made available on the network through an upgrade,” VP of Product and Engineering at Overclock Labs, Anil Murty, wrote.
This initiative paves the way for future advancements, with both teams eager to explore additional possibilities on Akash Network.
Hot Take: The Future of AI Training
As the collaboration between Overclock Labs and ThumperAI showcased the potential of training foundational AI models on Akash Network, the future holds promising advancements in the realm of decentralized cloud computing and AI integration.
By
By
By