Why AI won't replace programmers

In the era of rapid development of AI and LLM, I increasingly see that many people think AI will replace all programmers and they will not be needed. Even Nvidia CEO Jensen Huang expressed this idea.

But despite AI's impressive achievements in code generation, there is a fundamental limitation that prevents them from completely replacing humans in programming: their inability to truly reason.

Illusion of Intelligence

When we send a request to LLM, he seems to know the answers to absolutely all questions. It looks magical, but the reason is quite simple – large LLMs are actually trained on the entire Internet. It's hard to imagine, but the LLM training set actually includes absolutely the entire Internet. Because of this level of scale, it feels like the LLM is intelligent. But it is important to understand that this is just a retrieval of already existing solutions or their combinations, but not a solution to new problems.

Reasoning Test – Strawberry

A very simple example: How many r's are there in the word strawberry? Any person will say that there are 3 of them, but LLM answers – 2. The secret why ChatGPT incorrectly calculates the number of R letters lies in tokenization. Tokens can be thought of as the building blocks that the model uses to understand and generate text. The presence of tokens allows you to quickly and efficiently process text, but does not imply that the model has reasoning abilities.

In the GPT-4 tokenizer you can see in detail why this happens

From an AI perspective, “strawberry” is not a sequence of individual lettersthis is also not a series of lines [str, aw, berry]but rather sequence token IDs [496, 675, 15717]. LLMs, despite their apparent magic, should be thought of as a statistical modeling system that performs the task of predicting the next token based on training data. That is why she cannot count the number of r in the word strawberry/

Reasoning Test – Algorithmic Problem

Let’s take a simple and fairly old task to prepare for an algorithmic session at almost any IT company – Invert Binary Tree. This problem can be easily solved by an intern in almost any company, and in its classic form, all LLMs will also solve it.

• Let's add just a few nuances to make the problem new, which was not in the LLM training set.

  • It is necessary to invert the keys (permutation with bit reversal)

  • The solution must be a pure recursive function with no dependencies

  • The function must be of type Bit -> Tree -> Tree

Any programmer who knows how to code will solve the problem of adding these nuances. However, no LLM can solve it, not even the latest versions of chatGPT, Claude or Gemini. The solution is very simple and only contains 7 lines, but requires some reasoning ability.

These small changes are enough to take the problem out of the “memorized solutions zone.” It's not on the internet, and most importantly, all modern AIs can't handle it, no matter how you ask them.

Link to problem source: https://gist.github.com/VictorTaelin/45440a737e47b872d7505c6cda27b6aa

Why is this important?

1. Incapacity for original thinking: LLMs can combine existing solutions, but are unable to create truly new ones.

2. Research limitations: This inability to solve new problems means that LLMs will not be able to make original contributions to scientific research.

3. The illusion of understanding: LLMs give the impression of understanding, but in reality they only reproduce the combinations from their training set.

4. The limit of automation: While LLMs can automate many tasks, they cannot replace humans in creating truly innovative solutions.

It is worth noting that LLMs in any case provide a strong increase in the productivity of current programmers:

• They will significantly increase the productivity of programmers.

• Probably automate 99% of existing tasks.

• They will have an impact comparable to the advent of computers and the Internet.

Today, a key limitation remains: LLMs are incapable of true reasoning and the creation of original solutions.

Conclusion

Understanding the principles of modern artificial intelligence is more relevant than ever. A fundamental understanding of how AI works will help you critically evaluate the claims of AI product salespeople and filter their claims that AI will completely replace programmers.

Among the recommended materials for study I can highlight

1. Andrey Karpathy's channel

2. Stanford University Courses on AI and Machine Learning

3. Courses at the School of Higher Mathematics

4. Courses at the Faculty of Computer Science of the Higher School of Economics (FCS HSE)

Having a deep understanding of how AI works will help you better filter news and statements, especially those coming from people who don't have a deep understanding of the technology.