There are essentially two challenges for generative AI in enterprise space. The first is that every company wants a solution based on their private data, and the second is that the data is too sensitive for companies to publish on the web. Subtl.ai started building a solution for this back in 2020.
“Microsoft has a lot of security features on top of its OpenAI offerings. But at the end of the day, it’s still a common endpoint, making it difficult for companies in the banking or sensitive data sector to rely on it,” said Vishnu Ramesh, founder of Subtl.ai, in an interview with AIM.
Ramesh calls Subtl.ai a ‘private Perplexity built on light models for enterprise’. The company started as a for-profit research company out of IIIT Hyderabad and has built a light stack, which can sit on top of the existing cloud of the enterprise customers. It remains disconnected from the internet to protect the privacy of the data.
“The main reason we started this was to check if we can actually build something like Google from India,” said Ramesh, adding that many companies such as FreshWorks have built amazing products, but can something like Google come out of India. “In 2020, the vision was to make Google private, which slowly shifted to giving people a way to talk to their documents,” he added.
“Even with Google, there is a team of 1000 people sitting on the backend and verifying all the information that goes to the users, which is a very hard thing to do for 1000 enterprise customers.” With this in mind, Ramesh went on to acquire several defence contracts to build the product.
Perplexity recently released its Perplexity Pro for Enterprise, which offers similar solutions as Subtl.ai, but customers are worried about using it. Even though it gives real-time information with access to the internet, it is yet to crack the private use case for enterprise with regards to security.
Similarly, Atlassian has released Rovo, which offers similar capabilities for enterprise customers to allow workers to access data and information from external sources along with the data held within the enterprise.
Then, what is the moat?
OpenAI also offers retrieval capabilities through its API, and the same is the case with Google and Anthropic. But the recent Amazon Q case where the information was leaked to the internet has been on the minds of enterprise AI adopters. This has made executives seek solutions that stay off the internet, but still access the private data.
This is where Subtl.ai comes in.
In a demo presentation of the product set to be launched this week, Ramesh showed an example of how the State Bank of India is leveraging this solution. Built on top of Llama 3 8B and Subtl.ai’s proprietary retrieval solutions, the product can access all the information of catalogues and databases that the bank trained on and can reply to simple questions citing the exact source along with a paragraph.
“What would take 3-5 business days for a simple keyword query, our offering does in less than a minute,” quipped Ramesh, about the SBI offering. He also added that most of the data for SBI were the RBI regulations, which were publicly available on the internet. It was around 40% of the solutions that SBI operators required.
“There is value in getting focus paragraphs. It’s a great user experience for the customer.”
“This is what gives us the moat,” said Ramesh, adding that other offerings which are also connected to the internet provide a link and a PDF for private data as the cited source. It makes the task to search for the exact information another added task for the users. Subtl.ai gives a paragraph from a 600-page long PDF, citing the exact source of information for quick verification.
The future and issues
The only thing that falls short is that currently the offerings are only text based. When it comes to multimodal capabilities, Ramesh said that the data is converted to text from audios or videos and fed into the engine. In an example, he showed a video of all Kunal Shah podcasts, which were transcribed through YouTube and fed into the machine. If a user asks any question based on it, the chatbot generates the response giving the exact timestamp from the video where Shah mentioned the sentence.
Moreover, when it comes to multilingual capabilities, Ramesh said that they worked with Agastya Foundation, an NGO in Karnataka, where underprivileged kids would ask doubts in Kannada. These would be converted to English manually and fed into the system for answers.
“We are a small company and we do not have vast resources like OpenAI. But we are confident that our model can understand text and provide answers based on that. Everything else will just come around,” said Ramesh.
Ramesh narrated the story of a law firm that reached out to him for the offering. It made him curious about why someone would want the product when others are offering it. The reason is the same – giving exact information while also giving extracts from the documents, instead of PDFs and page numbers.
This also helps in reducing the hallucinations within the model as the information is being retrieved from small amounts of data sources instead of whole documents. Ramesh claims that Subtl.ai offers 75% lesser hallucination rate when compared to its competitors.
The company started with using OpenAI solutions, moved to Mistral, and now uses Llama 3 and has five models under the hood for seamless experience for its customers, with the second biggest model being only of 110 million parameters, making it very lightweight and easy for customers to integrate.
In the coming weeks, Subtl.ai plans to release the model for enterprise for free for a month and also give a private product on the internet for people to test out.
