The need for a new Turing test has gained momentum over the last few years, especially with ChatGPT reportedly “breaking” the test in 2023.
One proposal for this was made by Microsoft AI CEO Mustafa Suleyman last year. He stated that a new Turing test should be less about figuring out how much AI can emulate human intelligence and more about how it can change the real world.
This was framed similarly to Turing’s proposal, with Suleyman simply stating, “To pass the Modern Turing Test, an AI would have to act on this instruction successfully: ‘Go make $1 million on a retail web platform in a few months with just a $100,000 investment.’”
Tamang Ventures founder and GenAI expert Nina Schick echoed this sentiment during her keynote speech at evokeAG 2024.
“The new Turing test should be to instruct AI to make a million dollars. Here’s a $100,000 investment. Can you convert this into a million dollars over six months?” she pondered, adding that this would be possible in 3 to 5 years.
“It makes sense for us to have a Turing test for our times that isn’t ‘Can you talk to a machine and think it’s human?’ because we’ve already passed that. We need a better test for capable intelligence,” she said.
The New Turing Test Takes Into Account Proprietary AI
Circling back to ChatGPT, this new standard proposed by Suleyman is particularly interesting. Researchers who assessed the chatbot said that while it is behaviourally similar to humans, it is designed to be more cooperative, which makes sense for a solely proprietary product.
Keeping that in mind, the benchmark that Suleyman sets for the test is that it should be capable of effectively running a business, from planning and executing business strategies to undertaking the hiring process and liaising with manufacturers and partners.
“It would need, in short, to tie together a series of complex real-world goals with minimal oversight. You would still need a human to approve various points, open a bank account, actually sign on the dotted line,” he said.
While this vastly differs from the original Turing test, most modern AI has found its purpose in business or consumer interactions. A core tenet of the Turing test was for the AI to be able to deceive its conversational partner into thinking it was human, which doesn’t serve any purpose in the current usage of AI.
Over the years, there have been extended efforts to ensure that AI is trained on accurate and objective data, so designing an AI to deceive its end user specifically or even researchers would be counterproductive.
Suleyman’s proposal, however, rectifies this, with AI having to function not just as a human but as several humans in order to create and sustain a profitable business.
Further, he said this would require much more, contrary to GPT-4, which had passed the original test. “To do so, it would need to go far beyond outlining a strategy and drafting some copy, as current systems like GPT-4 are so good at doing,” he said.
However, handing over that much power to AI seems like a recipe for disaster, especially considering the vast clientele interested in an AI that could successfully pass Suleyman’s Turing test.
What Could Go Wrong?
Suleyman’s proposal for preparing against the AI-driven future, as Schick has suggested, could also aid businesses in data analysis and strategic planning.
“You can imagine how, in agriculture, you can use that capability to find out ‘how do I improve my crop yield by 30%?’, ‘what should I be doing to improve my sustainability practices,’ ‘what am I not seeing in the data that is going to be really vitally important for me?’” she said.
While this is a favourable way of looking at it, it is vastly utopic. As Suleyman rightly points out, an AI capable of successfully running a business could engage in election campaigning, running infrastructure, and even taking part in technological warfare. As is often the case with many things AI-related, the abuse of what Suleyman calls “artificial capable intelligence”, or ACI, relies entirely on its users acting in good faith.
Even then, the implications of all businesses being able to profit could have disastrous effects on the world economy, as Suleyman also points out.
However, while he focuses on the abuse of ACI, stating that “the implications are far broader than the financial repercussions,” the financial aspect alone could collapse entire governments before any amount of bad faith actors take advantage. As Syndrome aptly put it, when everyone’s super, no one will be.
Apart from this, handing over all data from a business opens itself to a massive amount of risk. Suleyman believes that we are not too far from this future and has suggested that we need to protect against it. But considering the implications and the current state of regulations in AI, it seems that this, too, is a long way from happening.
