Are the Days of the Human Programmer Numbered?

Are you a developer or data scientist with average good programming chops? Well, your job might not be as safe from the encroachment of AI as you thought.

For the longest time, the kind of problems that AI could solve was limited to relatively simple math and programming problems. But early this month, DeepMind announced that it had created an AI system that produces software code at a “competitive level” and on par with an average programmer.

The non-human programmer

With the permission of Codeforces, a popular platform that hosts regular coding competitions for participants to test their skills, the AlphaCode system participated in 10 different challenges, solving them well enough to be placed at about the level of the median competitor.

This is an impressive technical achievement, considering that Codeforces typically see tens of thousands of programmers from around the world who participate.

Goggle software engineer Petr Mitrichev lauded the achievement, pointing out that solving competitive programming problems is “really hard” as it requires both good coding skills and problem-solving creativity. (Both Google and DeepMind are subsidiaries of parent company Alphabet).

“I was very impressed that AlphaCode could make progress in this area, and excited to see how the model uses its statement understanding to produce code and guide its random exploration to create solutions,” Mitrichev said in comments reproduced in the AlphaCode blog post.

DeepMind’s AI Coding Engine

According to the AlphaCode team, solving the coding challenges entails moving beyond traditional AI strategies to writing novel code: “It’s not possible to solve problems through shortcuts like duplicating solutions seen before or trying out every potentially related algorithm. Instead, our model must create novel and interesting solutions.”

So how did the AlphaCode team pull off this feat?

In a nutshell, the team combined advances in large-scale transformer models with large-scale sampling and filtering. The AI Coding Engine was trained on selected public GitHub code and fine-tuned on a “relatively small” competitive programming dataset. The team then created multiple sets of programs for each problem, winnowing it down to a small set that was submitted.

“At evaluation time, we create a massive amount of C++ and Python programs for each problem, orders of magnitude larger than previous work. Then we filter, cluster, and [re-rank] those solutions to a small set of 10 candidate programs that we submit for external assessment. This automated system replaces competitors’ trial-and-error process of debugging, compiling, passing tests, and eventually submitting,” wrote the AlphaCode team.

The team promised that this is only a start: “Our exploration into code generation leaves vast room for improvement and hints at even more exciting ideas that could help programmers improve their productivity and open up the field to people who do not currently write code.”

Programming jobs are secure, for now

To be clear, it will be some time yet before AI can replace human programmers – if ever. For though AlphaCode could solve coding problems that are difficult in the context of a competition, the problems themselves are highly structured and hardly indicative of real-life coding tasks.

In fact, some senior developers are leery about coding challenges that are sometimes used when hiring programmers, pointing out that these programming problems often have limited relevance to the work they are hired to do on a day-to-day basis.

There is also the problem of insecure code to consider. For instance, the code generated by AI-powered GitHub Copilot, which we covered last year, was found to contain security vulnerabilities in as much as 40% of its output in a study published last December.

This likely came about from the presence of exploitable, buggy code that Copilot was unwittingly trained on, which in turn caused it to recommend insecure code. Unfortunately, this raises the possibility of bad actors deliberately introducing code with hidden backdoors into the public domain, which could then find its way into a new generation of AI models trained to write code.

Taken together, this probably means that experienced programmers will likely be needed for a long time yet, both to solve real-world problems, and to ensure that the code produced is secure.

And they might just have an easier time at it, too, as AI-powered tools are gradually introduced into coding platforms to help them write mundane code faster.

The pre-print from DeepMind is available here (pdf).

Paul Mah is the editor of DSAITrends. A former system administrator, programmer, and IT lecturer, he enjoys writing both code and prose. You can reach him at [email protected].​

Image credit: iStockphoto/Lightcome