A new type of AI could replace existing artificial neural networks and save energy

Learning with Light: This is what the dynamics of a light wave used inside a physical learning machine might look like. What is decisive is both its irregular shape and the fact that its development occurs in the opposite direction exactly from the moment of its greatest extent (red).

Artificial intelligence not only provides impressive performance, but also creates significant demand for energy. The more complex the tasks it is trained to solve, the more energy it consumes.

Victor Lopez-Pastor and Florian Marquardt, scientists from the Max Planck Institute for the Study of Light in Erlangen (Germany), suggest a method, with which you can train artificial intelligence much more effectively. Their approach is based on physical processes instead of the digital artificial neural networks currently used. The work was published in the journal Physical Review X.

Artificial intelligence (AI) company Open AI has not disclosed the amount of energy required to train GPT-3, which turns ChatGPT into an articulate and apparently well-informed chatbot. According to German statistics company Statista, this would require 1,000 megawatt-hours – about the same as what 200 German households with three or more people consume per year. Although this energy expenditure allowed GPT-3 to learn that the word “deep” in its data sets was most often followed by the word “sea” or “learning”, the general consensus is that it does not truly understand the underlying meaning of these phrases.

Neural networks on neuromorphic computers

With the goal of reducing the power consumption of computers and AI applications in particular, several research institutes have been exploring an entirely new concept for how computers will process data in the future over the past few years. This concept is known as neuromorphic computing. Although similar in name to artificial neural networks, it actually has little in common with them since artificial neural networks run on regular digital computers.

This means that the software, or rather the algorithm, is modeled on the principle of the brain, and the hardware is digital computers. They perform the neural network’s computational steps sequentially, one after the other, separating the processor and memory.

“Transferring data between these two components itself consumes a huge amount of energy when the neural network is training hundreds of billions of parameters, i.e. synapses, with data volumes of up to one terabyte,” says Marquardt, director of the Max Planck Institute for Light Science and professor University of Erlangen.

The human brain is designed very differently and would probably never become evolutionarily competitive if it operated at the same energy efficiency as silicon transistor computers. Most likely, it would fail due to overheating.

The brain is characterized by the fact that numerous stages of the thought process are not performed sequentially, but in parallel. Nerve cells, or rather synapses, are both a processor and a memory. Various systems are being considered as possible candidates for the creation of neuromorphic analogues of nerve cells in the world, including photonic circuits that use light rather than electrons for calculations. Their components simultaneously function as switches and memory cells.

Artificial intelligence as a hybrid of pinball and abacus: In this thought experiment, a blue, positively charged pinball represents the training data set. The ball is launched from one side of the plate to the other.

A self-learning physical machine independently optimizes the functioning of its synapses

Together with Lopez-Pastor, a doctoral student at the Max Planck Institute for the Study of Light, Marquardt developed an effective training method for neuromorphic computers. “We have developed the concept of a self-learning physical machine,” explains Florian Marquardt. “The basic idea is to conduct learning as a physical process, where the machine parameters are optimized by the process itself.”

When training conventional artificial neural networks, external feedback is needed to regulate the strength of many billions of synaptic connections. “The absence of this feedback makes learning much more effective,” Marquardt says. Implementing and training artificial intelligence on a self-learning physical machine will save not only energy, but also computing time.

When training conventional artificial neural networks, external feedback is needed to regulate the strength of many billions of synaptic connections. “The absence of this feedback makes learning much more effective,” Marquardt says. Implementing and training artificial intelligence on a self-learning physical machine will save not only energy, but also computing time.

“Our method works regardless of what physical process is happening in the self-learning machine, and we don’t even need to know its exact behavior,” explains Marquardt. “However, this process must satisfy several conditions. Most importantly, it must be reversible, i.e. it must be able to work in both the forward and reverse directions with minimal energy loss.”

“In addition, the physical process must be nonlinear, that is, quite complex,” says Marquardt. Only nonlinear processes can perform complex transformations between inputs and outputs. A ball rolling on a plate without colliding with another is a linear process. However, if he will be disturbed by another ball, the situation will become non-linear.

Practical test in optical neuromorphic computer

Examples of reversible nonlinear processes can also be found in optics. Thus, Lopez-Pastor and Marquardt are already collaborating with an experimental group developing an optical neuromorphic computer. This machine processes information in the form of superimposed light waves, with appropriate components adjusting the type and strength of interaction. The goal of the researchers is to put the concept of a self-learning physical machine into practice.

“We hope that in three years we will be able to present the first self-learning physical machine,” says Florian Marquardt. By this time, neural networks should appear that use a much larger number of synapses and are trained on much larger amounts of data than today.

As a result, there is likely to be an even greater desire to take neural networks beyond conventional digital computers and replace them with efficiently learning neuromorphic computers. “Therefore, we are confident that self-learning physical machines have every chance of being used in the further development of artificial intelligence,” says the physicist.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *