There was much interest in the possibilities around Metaverse captured during the pandemic as people were looking for more meaningful ways to connect with each other. However, since then, the hype around the metaverse has declined. As it turns out, the latest technology to capture public attention, Generative AI, can play a significant role in empowering interactions and development on the Metaverse. The creative applications of generative AI can fill in for the huge demands for content that developers need to create in virtual worlds with 3D assets.
Faster pace and more imaginative
Prajod Vettiyattil, Principal Architect at Fractal, explains just how. “Generative AI can help because it can create metaverse content from text descriptions. 3D modelling is normally labour-intensive work. There is a combination of skills required – you need to have some amount of artistic knowledge about colours and textures and their combinations it’s not just about creating the 3D model. Say, for a bank, a school, or a store, certain kinds of colour combinations and lighting that denote these themes must be envisioned and used. Then, some videos and images must be created and integrated into the 3D model”, he explains.
Generative AI makes it easier to create virtual spaces, and the customisations that generative AI tools offer can make the artistry much more appealing, expanding the options at hand. Currently, each space and experience in the metaverse is painstakingly created and curated by human designers and developers. It takes weeks, or sometimes even months, to generate each space.
“Generative AI can be a good tool while ideating and not just when creating the final environment. For example, many Generative AI tools used for 2D visualisations like MidJourney and DALL.E 2, can be prompted to emulate the style of any artist you choose, like Van Gogh”, Vineet, Senior Consultant at Fractal stated.
“The power to create avatars in the metaverse is the most interesting and complex aspect of the metaverse. An avatar will be a bot that can move, talk, and interact like a human. An avatar can be a representation of a human existing in the real world or a virtual-only being with no representation in the real world. We can use Generative AI to simulate realistic thoughts, physical appearance, voice, and movements in the metaverse. Generative AI can help build AI bots with emotional and conversational characteristics that are personalised for your preferences, much like the AI Virtual Assistant, in the film “Her”, Prajod said.
However, it’s the simplicity with which generative AI does it, that makes the world of difference. Prajod explained, “The key point in the value of generative AI is that automating the creation of interactive content and experiences becomes very easy. For example, making animated films like Avatar takes years because of hundreds and hundreds of hours of animation work and many iterations that go into it. But with generative AI, you can do it in months”.
Enhanced immersion
Vineet believes that these applications can step up the overall quality of social interaction through their different modalities. “Just like voice communication was a huge transformation after letters and telegrams. And the video was another big step in interactivity. You can use generative AI to analyse ML algorithms or to create a piece of music and make it sound like Mozart, who had many unfinished symphonies before he passed away. So, using that logic, you can even make music stylistically like maybe ‘Lord of the Rings’ or ‘Led Zeppelin’. Just like ChatGPT can be used to build narratives; in the future, generative AI can be used in other modes like audio” he explained.
All this is to say that the metaverse can finally become the fully immersive experience that it promised to be. Prajod further shared, “Unlike the text interface of chat apps and 2D interface of video conference apps, metaverse allows kinesthetic interactions like turning your head and moving your hands. This is a more immersive social experience compared to text or video”.
Adds great value to financial and educational sectors
Notably, it isn’t just for entertainment, there’s value to be found in the Metaverse through Generative AI for business and education as well. “This technology opens new opportunities not just in human interaction and entertainment but also in education. There is no need to put in any money in physically building classrooms and other facilities and scaling in these sectors can become far easier”, Prajod said.
Businesses such as banking, which are already leveraging the metaverse, can do more to help themselves. “Now, there’s a lot more you can do in terms of physical virtual interactions other than just automated chat, especially in the banking sector. There are virtual banks which eliminate the cost of physically going there or even the hassle of contacting someone at a call centre and waiting on phone lines.”
He adds, “Instead, even banks can invest their resources into other places. Even if you have a million customers, you can have a customer service representative for each of them because they’re just virtual avatars. It’s not just audio, there will be visual aspects which is a much fuller experience because you’d be able to see their expressions when they’re talking. This is valuable when you’re selling services to customers compared to calling them over the phone”.
Much like human customer support assistants, these bots can access internal data and respond to customer queries. Furthermore, there will no longer be a never-ending wait on phone lines when such virtual customer support assistants are deployed. These AI bots can also learn from the customer’s responses and get better at explaining and conversing based on each customer’s history.
“If I am calling a customer representative normally, a lot of the times I am on call with someone else the second time I call, so, the context is lost. But, on the metaverse that’s not a problem because the context can be retained when you’re talking to a chatbot”, Prajod stated.
But this isn’t the end of the positives. “There’s a lot of scopes, especially to create content for visually impaired users because of auto translation. You can also converse with people across the world in real-time using the auto-translate speech feature”, he further explains.
Other applications
In terms of applications, digital twin is another space with underlying tech similar to the metaverse where generative AI can help. “Digital Twin technology needs high-resolution rendering and real-time data transfer. We are very excited about digital twins, which are replicas of certain processes or structures which can be connected to IoT (Internet of Things) endpoints. If you are creating a digital twin of, say, New York or Mumbai, you can show the virtual spaces of the roads and signals, and you can solve real-world problems of how to control the flow of traffic and showcase alternative routes of getting to places faster”, Vineet stated.
There are other challenges around the metaverse which generative AI can resolve. “The bandwidth needed for a customer to be able to experience the metaverse is a lot. Power and computing are considerable limiting factors currently. In this regard, generative AI can be used to re-generate the content on customer’s edge devices making it possible to show high-resolution content. Currently, most metaverses have a cartoonish look and a low resolution”, Prajod shared.
