Narendra Sen, the CEO of NeevCloud, is a man of ambition. He envisions constructing an AI cloud infrastructure tailored for Indian clients, comprising 40,000 graphics processing units (GPUs) by 2026. This infrastructure aims to support Indian enterprises with training, inference, and other AI workloads.
Yotta, another prominent AI cloud provider, has recently gained attention for its recognition as the inaugural NVIDIA Partner Network (NPN) cloud partner in India, achieving the Elite Partner status globally.
Yotta’s ambitious plan is to set up an AI cloud infrastructure with 32,768 GPUs by the end of 2025. NeevCloud wants to do better.
Moreover, NeevCloud will soon launch an AI inferencing platform which will provide open-source models like Llama 3 series, Mistral models and DBRX by Databricks.
“Later this month, we plan to introduce the DBRX because we see a great demand for the model in this country. Subsequently, we will launch the text-to-image models from Stability AI.
“We are focusing on a step-by-step approach to ensure smooth operation. User experience is paramount, and if everything proceeds as planned, we might expand the range. Meanwhile, we are also enhancing our capabilities in tandem with these developments,” Sen told AIM.
AI Inference
Inferencing involves applying the pre-trained model to new input data to make predictions or decisions. So far, around 3,500 developers have signed up to use NeevCloud’s inference platform.
The company plans to launch the beta version of the platform this month and provide developers with free tokens to lure them to come and test NeevCloud’s inference platform.
For the inferencing platform, the company plans to leverage around 100 NVIDIA GPUs, including NVIDIA H100s, A100, and L40 GPUs. Moreover, NeevCloud plans to introduce AMD’s M1300X to the mix.
“AMD’s M1300X provides cost benefits compared to the rest. Indians don’t care about the brand. What they care about is that the API should work, latency should be fast, and they should get immediate tokens – that’s it,” Sen stated.
NeevCloud could also be the first company to bring Groq’s language processing units (LPU) to India. Recently, Sen posted a photo of him on LinkedIn, posing with Jonathan Ross in Groq’s headquarters in San Francisco.
( Source: LinkedIn)
Sen, however, refrained from revealing much in this regard as things are still not finalised. “While we all know the big names like NVIDIA, there are other players in the market that we are trying to onboard, like SambaNova,” he said.
AI Infrastructure as a Service
For the AI cloud, Sen revealed that they have placed an order with HP Enterprise (HPE) for 8000 NVIDIA GPUs, which they are expecting to receive in the second half of this year.
NeevCloud will compete directly with Yotta in the AI-as-a-infrastructure space. Notably, another company that has already making GPUs more accessible in India is E2E Network.
The NSE-listed company offers NVIDIA’s H100 GPU and NVIDIA A100 Tensor Core GPUs at a competitive price compared to large hyperscalers.
Recently, Bhavish Aggarwal, the founder of Ola Cabs, announced his decision to offer Krutrim AI Cloud to Indian developers.
Similarly, Tata Communications also partnered with NVIDIA to build a large-scale AI cloud infrastructure for its customers in the private as well as the public sector.
While Krutrim and Tata Communications have not revealed the number of GPUs they plan to deploy, NeevCloud plans to deploy another 12,000-15,000 GPUs by 2025. “Then, by 2026 we will deploy the remaining to hit the 40,000 GPUs target,” Sen said.
How will NeevCloud Fund the 40,000 GPU Acquisition?
However, deploying a GPU cluster of around 40,000 GPUs will cost billions of dollars. According to Sen, it will cost them approximately $1.5 billion.
While Yotta is backed by the Hiranandani Group, NeevCloud is banking on its data centre partners to help not only procure but deploy the GPUs as well.
So far, NeevCloud has partnered with three large data centre companies in India, two of which are based in Chennai and one in Mumbai. One among them is one of the largest data centre operators in India, Sen said.
“What we have in place is a revenue-sharing model. While they already have the data centre infrastructure, they need to deploy the GPUs on our behalf, and NeevCloud will bring in the customers, who will access both their AI cloud capacity (to be deployed) and their data centre servers,” Sen said.
Sen established NeevCloud in 2023. However, Sen has been running a data centre company in Indore called Rackbank Datacenters for many years now.
Located in Crystal IT Park, Indore, the data centre is 35,000 sq ft and has a capacity of 32,000+ servers.
The company has innovated a liquid immersion cooling technology, named Varuna, to effectively cool high-density computing hardware utilised for AI, machine learning, and high-performance computing (HPC) tasks.
This method entails immersing servers and other IT equipment in a dielectric, non-conductive coolant liquid, facilitating direct heat transfer from the components to the liquid.
