Recently, Oracle released HeatWave GenAI, touted as the industry’s first in-database LLM. Embedded within the MySQL database, the LLM eliminates the need for separate infrastructure or complex integration steps. The biggest positive here is that HeatWave doesn’t need GPUs for operations.
“We don’t need a GPU at all. We already fine-tuned it, built into the model, built into MySQL. So there is no need for a GPU to further train it,” said Palanivel Saravanan, vice president of cloud engineering at Oracle India, in an exclusive interaction with AIM.
GPUs are considered as the most scarce resource when one is looking to build an LLM. However, Saravanan reiterated, “There is no GPU in the process. It’s not elimination, there’s just no requirement of GPUs because we have already optimised the language model here.”
LLM with Unstructured Data
MySQL HeatWave integrates models like Llama-3 and Mistral, simplifying deployment. Its native VectorStore manages unstructured data, improving accuracy and efficiency in generating statements and performing analytics. HeatWave streamlines these processes without extensive manual adjustments.
The HeatWave setup enhances accuracy by organising data into binary vectors that establish relationships. It accommodates various data types, ensuring robust model readiness without extensive modifications.
Saravanan further explained that any customer-specific document updates go directly to the automated vector store, organising data efficiently alongside existing information. With LLM residing in the database, there is a significant improvement in speed, security and even cost-efficiency.
“Today, if you want to build a large language model for commercial use, you have to follow 10 steps. But over here, it’s just two steps,” said Saravanan.
Interestingly, the importance of vector databases have significantly gone up with generative AI. Vector databases provide LLMs with access to real-time proprietary data thereby assisting with development of RAG applications, which is an integral part of HeatWave.
Special Purpose LLMs
With HeatWave GenAI, Oracle has built a special purpose LLM, something that Saravanan believes enterprises prefer over general-purpose LLMs. “Every enterprise looks for a general purpose large language model, maybe for evaluation purposes, but then to align to the business or to map to the business, they need a very special purpose LLM,” he said.
With specialised LLMs, enterprise adoption would also be on the higher side. Interestingly, a number of companies have announced small language models which are a form of special purpose models.
Recently, AI expert and innovator Andrej Karpathy said, “The LLM model size competition is intensifying… backwards,” alluding to the rise and efforts of big tech companies to build small language models.
Last week, OpenAI released the GPT-4o mini, and Mistral announced Mathstral, a 7B parameter small language model.
HeatWave, the special purpose model by Oracle is said to be 30x faster than Snowflake, 18x faster than Google BigQuery, and 15x faster than Databricks for vector processing.
Cloud Prowess In India
Continuing on building special purpose models, Oracle announced a specialised data cloud architecture. A few weeks ago, the company officially announced Exadata Exascale, which is the world’s only intelligent data architecture for the cloud, that provides extreme performance for all Oracle database workloads, including AI vector processing, analytics, and transactions.
“Exadata Exascale offers Indian businesses a significant reduction in infrastructure costs by up to 95%. Our smart, scalable, and secure infrastructure allows organisations to pay only for what they use, improving operational efficiency,” said Saravanan.
Exascale’s intelligent data architecture works on pay-per-use economics, allowing enterprises of any size to use the platform as per their need. It features a virtualised, database-optimised infrastructure with shared compute and storage pools.
The company has been on a high with promising revenue numbers, riding on its cloud and generative AI services integrated on them. OpenAI had also decided to run its workloads on OCI, extending the Microsoft Azure AI platform to Oracle’s cloud services.
