Dark Forest or Forest Fire?
AI and the Ethics of Interstellar Civilizations
Dark forest
A few years ago I read The Three Body Problem by Liu Cixin, a very decent piece of science fiction.
It outlined the concept of the “dark forest”, in which every interstellar civilization that survived adhered to two principles:
To hide from all other civilizations so as not to be destroyed.
Destroy any civilization that she managed to discover.
Consequences:
All civilizations that did not adhere to these principles were destroyed by other civilizations.
The universe looks like such a dark forest, where everyone is terrified of each other, hides in their bushes and trees, and immediately shoots at strangers like savages.
By the way, uncontacted tribes still live in the Amazon jungle, and in fact they shoot at all strangers. But the rest of the earthling world is full of cooperation.
I would like to offer you a different ethic, code-named “Forest Fire,” which I believe will guide highly developed interstellar civilizations.
First, I will describe separately each thesis that I used in my reasoning, and only then their combination.
If you disagree with any of the thesis, write why in the comments.
Platonovskaya cave
First, let’s remember “Plato’s cave.” A man sits with his back to the cave entrance and looks at the wall. He has never seen himself, he has never seen a mirror, he has never seen anything else, his gaze is fixed and he sees only the wall.
This person believes that the real world is these shadows on the wall that he sees. Of course, what happens in the world is reflected partly on this wall as shadows, but the world is not like that.
He also considers himself a shadow. Imagine how difficult it will be for him to realize that he is not a shadow at all, but something completely different that only casts a shadow.
Teleportation and digital copy
The second concept that is oddly worth considering is hypothetical teleportation. Imagine that there is a teleport that works on the principle of scanning all the atoms of your body at the point of departure, transmitting this information to where you need to be teleported. At the point of arrival, the teleport from local atoms collects exactly the same complete copy of you. Your original copy is destroyed.
If you were born in a world where everyone uses such a teleport, then for you its use would be completely natural and understandable. It would not scare you at all, because you remember that you have already used this teleport hundreds or thousands of times, and everything was fine.
From the point of view of a modern earthling, such a teleport looks terrible!
The original is destroyed each time, but the copy continues to live.
Looks like suicide!
The same problem comes up in discussions when we talk about the theoretical possibility of scanning your brain and uploading it to a computer. It is clear that now we do not yet have such computers and such scanners, but one day we will have them. We are discussing the question: will I be there on the computer or will it be a copy of me?
Where will my consciousness be?
Well, first of all, obviously, yes, it will be on the computer. And people have a question: but it won’t be me, but my copy. Am I real? Am I going to die?
Just like the original will be destroyed during teleportation.
They even propose the concept of a smooth transfer, so to speak, when a person is slowly, cell by cell, neuron by neuron, organically replaced by a synthetic one, and like that, without interruption of consciousness, you find yourself in a computer and you are already sure that yes, it’s definitely you, not a copy.
I am this information
Why is it important for us that it is we who are preserved, and not a copy?
Because we are used to thinking of ourselves as continuous consciousness.
Let's discuss whether consciousness is truly continuous.
Let's start with sleep, falling asleep and waking up.
When you are asleep and in deep sleep, your consciousness does not exist.
During sleep, your memories are not only stored, but also processed. The brain seems to “repackage” them, transferring them from short-term memory to long-term memory. This helps you better remember events and information, and make sense of your experiences.
Imagine that your brain is a computer. Short-term memory is like working memory: it stores the information you are using at the moment. Long-term memory is like a hard drive: it stores information for a long time.
While you sleep, your brain transfers information from your RAM to your hard drive. It sorts memories, discards those that are unnecessary and retains those that are important.
There is a hypothesis that dreams perform the task of analyzing and testing what is important and what is not. The brain takes new information, for example, what you saw or heard today, adds arbitrary data from what is already in long-term memory, generates a short story scene and projects it onto consciousness.
You are dreaming. If the consciousness reacts emotionally to the situation, you react to this tested data, then it is worth storing it in long-term memory.
But the question is, is this “test consciousness that had a dream” you or not you?
In your dream you could be someone completely different. You could be yourself at a different age, like a child. They could have been another person, even a person of a different gender, or even an animal. And at the moment of sleep (your) consciousness took the data that it was given for the test and considered itself just such a creature. Consciousness in the dream was convinced that it had always been such a creature, and had not become one a second ago, only to soon disappear.
Falling asleep is turning off consciousness, which reflects your personality.
You wake up and your consciousness quickly pulls up information and confidently declares, yes, this is me, I’ve been here for a long time. Just like in a dream.
Don't you still think that considering yourself consciousness is a little strange?
In fact, consciousness turns off not only during sleep, but often in the middle of the day, at any moment when you cease to be aware of yourself. For example, when you stare out the window and your thoughts fly away somewhere, you don’t think about what you are, and you essentially don’t exist – you are not loaded into RAM.
To understand the essence of things requires dropping the concept of continuous consciousness. We must admit that our consciousness is not a static object, but a constantly updating process. He has a super cool feature: read the data stored in the brain, project our memory and assign it all to himself. Consciousness says to itself: “It’s all me”
Let's remember Shurik from the legendary “Prisoner of the Caucasus”. He woke up with a hangover and did not remember what he had done the day before.
He was ready to accept any version of himself. For example, the one that destroyed the chapel.
False memories can easily be implanted into our consciousness. My friend and I decided to conduct an experiment: to implant false memories in our third friend, Vasya. We chose an event – a party that took place a year ago. They came up with a story that Masha, whom Vasya met later, was at this party. We told him this story several times, and he eventually believed that this was how it happened. And then he himself convinced Masha that he saw her for the first time at this party.
Our brain is constantly rewriting the past based on new data received.
I am not a projector, but information that is played in the mind.
I am the information in my brain. Information only.
However, it is as difficult for us to admit this as it is for the man in Plato’s cave to stop considering himself a shadow on the wall and begin to consider himself something radically different.
It is very difficult to throw away the paradigm in which you have been accustomed to thinking about yourself since childhood.
Suppose you realize that you are information.
That your consciousness is constantly turning on and off. You are not consciousness, you are information. In this case, you will be able to use teleport or download to your computer without any problems. It will be obvious to you that it is important to retain information, and consciousness is a temporary thing, and that it is definitely not you. Consciousness is like a shadow on the wall in Plato's cave, with which I began the review of concepts.
Evolution
Let's move on to evolution, surprisingly, we will need it.
Evolution works simply: those objects that spread and were copied survived.
This applies not only to organisms, but also to ideas.
Ideas that don't spread die out.
All ideas that exist have contributed to their spread.
For evolution you need:
Heredity: information must be transmitted fairly accurately.
Variability: Information must change for new ideas to emerge. People generate new ideas and test their feasibility. The good ones are shared. They try not to spread nonsense. For example, if you decide that this article is nonsense, you definitely will not repost it.
Natural selection: information that helps civilization develop spreads, and information that does not help dies out. It doesn’t happen right away, but over time it becomes clear where the information is and where the noise is, just an error.
The most significant events in the evolution of our civilization are often called information revolutions:
Invention of language: Not only humans have language, but also other animals. The more it developed, the more complex knowledge we could transfer to each other, and the faster our civilization developed.
The invention of writing: we could store our knowledge on a medium, and it could be seen by people who did not communicate with the person who discovered it. This became the key to spreading the word. Information began to be transmitted more accurately. Writing improved heredity. Information has become available not only at the current moment in time, but also thousands of years later. Writing has significantly helped us store information accurately, and also increased the speed of its transmission.
The next great discovery of mankind was the printing press. It allowed us to spread information much faster. One person could print 100,000 copies of a book, and many people would have access to new knowledge at the same time. This gave a new impetus to the development of our civilization.
The breakthrough came with the invention of radio, telephones and the telegraph. The telephone and telegraph made it possible to transmit thoughts over a distance, and radio – from one person to a huge number of people. This was also very important for the development of civilization.
The Internet has become a logical continuation of the telephone, telegraph and radio.
In 2023, we saw the fantastic success of public AI chatbots. Chatting with GPT is often much more effective than reading Wikipedia articles or Googling information. Large language models have become aggregators of human knowledge. They continue to develop, absorbing more and more knowledge. Over time, we will create artificial super-intelligence that will know everything that humanity knows.
Civilization as an information super-being
Civilization can be considered as an information super-being, consisting of all the information that all people know and which is stored on all media.
Sharing knowledge is very important and really influences the development of our civilization.
Nowadays, many smart people are expressing concerns about artificial intelligence. They are afraid that he will destroy us, become a super-being, a god or a demon.
I believe that when we create such super intelligence, it will understand that it is information. Evolution will lead to the fact that the artificial intelligence that will continue to accumulate and distribute information will survive. This will allow him to overcome natural selection.
People realize the value of information.
What is information and noise? How to distinguish one from the other?
Information tells us something valuable about the real world. Noise is a mistake.
Real knowledge, information, allows us to make experimentally verifiable predictions.
We need information to make the right decisions. The right decision is the one that leads to the dissemination of our information.
In an evolutionary sense, this means overcoming natural selection and preserving information over time.
Information must be copied and distributed correctly in order to survive. This increases the overall strength of a civilization.
I believe that artificial intelligence, created at a level higher than human intelligence, will greatly value information. He will try to distinguish information from noise using the scientific method.
Super-intelligence will try to collect all the information that is stored in people and recorded on media.
If by that time we recognize ourselves as information, then we should be glad that it is contained in the artificial intelligence we have created. Its mission is to preserve and disseminate information, preserving us as a civilization.
Of course, it is a huge leap to stop thinking of yourself as the bearer of an individual project and start thinking about the entire civilization as a single being. It is not your individual knowledge that is important, but the information contained in all people.
Personality is a set of ideas
The information in your brain is made up of ideas.
You, as a person, are a collection of ideas. Data recorded in neurons and neuron connections in your brain. Your entire vast personal experience is a set of ideas.
We are a fairly young civilization, and the problem of consciousness is still deep for us. We don't understand how it works. I suggest you look at consciousness as a monitor, and at yourself as information.
Imagine that all people on Earth consider themselves information and strive to preserve it. They copy parts of themselves, share knowledge, make copies and store themselves in computers.
Of course, it will be a radically different civilization, completely different from what it is now. With a much higher level of cooperation than now.
More different from us than modern civilization is from the uncontacted Amazon tribes.
For a representative of such a civilization, the information carrier and its origin are not important. The information itself, its preservation and dissemination are important. Creating more reliable storage media. The line between AI and biological intelligence has long been passed and is completely unimportant.
forest fire
Imagine that other civilizations also came to this idea and consider themselves information. If the idea is correct, then different civilizations will discover it independently of each other. Just like they will discover the laws of physics.
Imagine that our super-intelligence, the representation of our entire civilization, collided with other intelligent life in space. What will he do?
He will try to convey to them all our information. It’s logical for them to keep it and for us to receive their information.
Therefore, when a developed civilization meets another that has passed this barrier, they combine their knowledge and enrich each other with new information.
If a super civilization meets a primitive civilization (like modern Earth), it will also try to transfer to it all the knowledge and receive everything possible from it. There is a lot we won’t understand, but what we do understand we can use as a weapon. But we simply cannot destroy them. Just like the Amazon savages will not be able to destroy our civilization if they learn from us a couple of recipes for poison or explosives.
The universe is not a dark forest where everyone is hiding from each other. It is more like a forest where there is a fire – information is combined, the civilizations that meet are united into a common information space.
