DeepMind’s AlphaCode has emerged as one of the hottest open-source code generators, besides OpenAI’s Codex, Tabnine, CodeWhisperer, and others. If you are new to programming, these tools will make you stand out from the crowd.
In February this year, the developers at DeepMind examined the potential of AlphaCode by testing it on competitive programming websites where human developers are given programming problems and ranked on the basis of their results.
Can Big Data Analytics come to the rescue in these turbulent times? Register >>
One of them was a competitive coding competition on Codeforces, a popular platform for hosting coding competitions. A selection of ten varied test problems from different stages of development was given to AlphaCode. The AI tool achieved an estimated rank within the top 54 percentile of participants that attended the contest, thus proving that AlphaCode’s code generation system has achieved results at a competitive level.
AlphaCode versus the world
AlphaCode’s AI system is pre-trained in various programming languages that include C++, C#, Go, Java, JavaScript, Lua, PHP, TypeScript, Ruby, Scala, Rust and Python. This dataset consists of approximately 715 GB of codes along with their descriptions.
Through AlphaCode, DeepMind has been able to fill the gap that is lacking in AI models like Codex—problem-solving skills. AlphaCode has not only been trained to “understand” natural language but also to design complex programmes and algorithms and implement them in code. It has also been trained to convert problem descriptions into code using thousands of problems from coding competitions.
When presented with a fresh problem, AlphaCode generates candidate code solutions (in Python or C++) compared to GPT-3 algorithms and filters out the bad ones. Whereas researchers had previously used models like Codex to generate tens or hundreds of candidates, DeepMind had AlphaCode generate up to more than 1 million.
Alphacode versus GitHub Copilot
OpenAI’s AI code suggestion tool GitHub Copilot runs on the natural language processing (NLP) model Codex, a boosted version of GPT-3. While it is built with the vision to achieve goals that are similar to that of AlphaCode, Copilot seems to have a difficult road ahead. Here are some of the differences between the two code generation tools:
- Training: GitHub Copilot’s AI Codex is trained to identify 12 billion parameters as compared to AlphaCode’s AI-based code generation model, which is trained with 40 billion parameters. This improves AlphaCode’s performance fourfold.
- Suggestion versus Generation: While GitHub Copilot is built to assist programmers in writing rudimentary sections of code, AlphaCode is capable of generating complete complex programmes.
- Complexity: While both AI tools are in the beginning stages of development, GitHub Copilot suggests basic code involving simple logic, whereas AlphaCode is tested to produce complex algorithms at a competitive level.
Expectations versus reality
Despite Alphacode performing well in coding competitions, these are usually focused on solving programming problems and giving solutions in the least time possible. Thus, this may not be a fair measure of real time usage.
Most real world problems need realistic and contextual solutions which need developers and programmers to think and solve complex problems, design systems, make design choices, provide different solutions and more. So, in essence, Alphacode might be able to implement coding for already known solutions reasonably well but will likely fail to perform when it comes to solving real world engineering problems.
So, it can implement but not invent!
Other platforms
Recently, OpenAI’s ChatGPT has also been able to solve some programming puzzles. Robert Sweeney, a tech entrepreneur, shared in a LinkedIn post that the chatbot was able to get a working solution with an explanation for a Leetcode question in less than ten seconds!
However, Ritesh Menon, a tech expert, revealed in another post that ChatGPT’s responses are articulate but not always accurate. Menon pointed out that while the tool does correctly solve some easy programming puzzles, it falters when faced with more difficult problems. In addition, it gives bizarre answers with equally plausible explanations when one hits the ‘ try again’ option against its responses.
Besides these, Salesforce’s Codegen is also another tool for coders. It is a 16-billion parameter, auto-regressive language model trained on a large corpus of natural and programming languages. Codegen can handle simple coding tasks but not so much when it comes to more complex problems.
In June 2022, Amazon also released CodeWhisperer, a machine learning-powered service that helps improve developer productivity by generating code recommendations based on developers’ comments in natural language and their code in the integrated development environment. Amazon claims that developers can speed up the development process with CodeWhisperer by simply writing a comment in their IDE’s code editor.
This month, Google Labs also unveiled Pitchfork, an AI that can convert old code to new code and rewrite itself. The list just goes on.
While these systems attest to what is possible with AI today in the realm of computer programming, only a few of such systems are open source. In an effort to change this, AI startup Hugging Face and ServiceNow Research R&D division launched BigCode, a new project that aims to develop “state-of-the-art” AI systems for code in an “open and responsible” way.
The BigCode team said that the goal is to eventually release a dataset large enough to train a code-generating system, which will then be used to create a prototype—a 15-billion-parameter model, larger in size than Codex (12 billion parameters) but smaller than AlphaCode (~41.4 billion parameters)—using ServiceNow’s in-house graphics card cluster.
It’s not always rainbows and butterflies for auto-code generating platforms. In November, Kite, an AI assisting tool that helps developers write code, announced that it would no longer be operational. The platform augments the coding environment with the internet’s programming knowledge and machine learning—similar to GitHub Copilot, which uses OpenAI CodeX, a version of GPT-3 language model.
