The story of how I tried to create an anonymous network on one of the cheapest Orange Pis

Introduction

Three or four years ago I was interested in single boards, microcomputers, mini PCs, etc. things. It was convenient to transport such computers when traveling, to move them when moving, and even if you take mini PCs, then they can be quite correctly and normally used as a main PC if there are no goals to do design, program for Android, play modern games or write them at all. But from that long and good time I still have orange pi’s, which I didn’t even really use, but bought them simply for the sake of something future and abstractly interesting.

But recently, while writing tests for my anonymous network, I suddenly remembered that somewhere in one of my shelves I had a cheap single-board device lying around. After searching a bit, I finally found it. It was Orange Pi Lite with 512MiB of memory and Wi-Fi module. Then it cost about 1.5k rubles. At that time, if my memory serves me correctly, it was generally the cheapest copy of the Orange Pi line, on a par with the similar Lite, only instead of a Wi-Fi module there was an Ethernet port.

As soon as I finally received the Orange Pi Lite, I immediately had the idea to test my anonymous network on this little animal. The essence of this idea lay on several levels: 1) the anonymous network, or rather the node in the anonymous network, did not consume more than 200MiB RAM (about 60MiB on average), which fit quite well into the single-board memory, 2) the anonymous network has already been tested on the cheapest VPS with weak processors ~2-2.2GHz, one core and small memory of 512MiB. 3) golang, as the language in which the anonymous network was written, is cross-compiled and cross-platform, as a result of which I did not need to adapt ready-made programs written on x86-64 for ARM, but could compile for any platform. In other words, during all of the above, I had every chance of success.

Plus, in addition to all this, the final solution also had an application use, namely, for the Orange Pi to constantly generate traffic on an anonymous network, and so that I could send and receive messages through it. In other words, the Orange Pi in the local network concept was supposed to be for me constantly running server which I could turn to constantly. And let’s see what came of it.

Setting it up

The entire setup came down to literally three parts: 1) installing the equipment, 2) installing Linux, and 3) installing an anonymous network. With the first point, everything is simple – connect the keyboard, monitor, prepare an SD card with Linux installed. For the second one, I simply downloaded the ubuntu distribution designed specifically for H3 ARM processor (installation instructions with all links) and using balenaEtcher installed this distribution on the SD card. At the beginning, out of stupidity, I simply downloaded random distributions, but I received either errors of not finding Wi-Fi drivers, then loading the distribution itself ended with errors, or the distribution did not load at all. After successfully installing Linux, it’s time to install an anonymous network node Hidden Lake. You can find out more about the anonymous network itself here, and about the full setup here.

In order to install an anonymous network node, a sufficient condition is to launch the program H.L.S. (Hidden Lake Service). I was specially prepared for this task script quick installation of HLS from the latest release version. But then I remembered that the script was designed for the architecture amd64but obviously not arm. But prepare release versions for the architecture armv6l for testing purposes, it was clearly more problematic than simply installing golang on the orange solder itself and compiling from there from HLS sources. This is what I did.

git clone https://github.com/number571/go-peer # качаем библиотеку go-peer в cmd/ которой присутствует hidden_lake
wget https://go.dev/dl/go1.16.linux-armv6l.tar.gz
tar -C /opt -xzf go1.16.linux-armv6l.tar.gz
echo PATH="${PATH}:/opt/go/bin" >> ~/.bashrc
source ~/.bashrc
cd go-peer/cmd/hidden_lake/service && go build ./cmd/hls
mv go-peer/cmd/hidden_lake/service/hls .

Next, we simply install and launch HLS as a service.

echo "
[Unit]
Description=HiddenLakeService

[Service]
ExecStart=/root/hls -path=/root -key=/root/priv.key
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
" > /etc/systemd/system/hidden_lake_service.service
systemctl daemon-reload
systemctl enable hidden_lake_service.service
systemctl restart hidden_lake_service.service

When the service is started, it will create two files: hls.cfg, priv.key and one directory: hls.db. A directory is simply a storage of hashes of received and sent messages. Such a storage performs one single role – checking and ignoring previously received or generated hashes. As a result, the circulation of the same information in the network is eliminated, and the use of one message more than once, even after a long interval of time, is eliminated. File priv.key – this is the generated private key from which HLS begins to be identified by the network as a valid node. File hls.cfg – this is the configuration itself, and it’s what interests us. So far, HLS generates garbage traffic only for itself, but does not send it to anyone and actually does not receive anything from the outside. We need to connect HLS to the global network. To do this, simply edit hls.cfg (add connections, delete address.tcp, delete backup_connections), for example, using the nano editor. Operating nodes (or rather repeaters) can be found in the table README.

{
	"settings": {
		"message_size_bytes": 8192,
		"work_size_bits": 20,
		"key_size_bits": 4096,
		"queue_period_ms": 5000,
		"limit_void_size_bytes": 4096,
		"messages_capacity": 2048
	},
	"logging": [
		"info",
		"warn",
		"erro"
	],
	"address": {
		"http": "127.0.0.1:9572"
	},
    "connections": [
        "195.133.1.126:9581",
        "94.103.91.81:9581"
    ],
	"services": {
		"go-peer/hidden-lake-messenger": "127.0.0.1:9592"
	}
}

After editing the config, simply restart the service: systemctl restart hidden_lake_service.service. We wait a little and check the work through systemctl status hidden_lake_service.service.

It works, but it’s not enough for us. Our main goal is not just to run a node on the Hidden Lake anonymous network, but also to use this anonymous network. There is currently only one application in HL – the HLM messenger. This messenger presents a GUI through HTTP server. Due to this property, this messenger can be used from other devices simply by tapping on the local IP and the desired port. Therefore, we need to also run HLM as a separate service.

cd go-peer/cmd/hidden_lake/messenger && go build ./cmd/hlm
mv go-peer/cmd/hidden_lake/service/hlm .

echo "
[Unit]
Description=HiddenLakeMessenger

[Service]
ExecStart=/root/hlm -path=/root
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
" > /etc/systemd/system/hidden_lake_messenger.service
systemctl daemon-reload
systemctl enable hidden_lake_messenger.service
systemctl restart hidden_lake_messenger.service

When launched, one file will be created: hlm.cfg and one directory hlm.db. A directory is a database of all sent and received messages. Here we are more interested in the hlm.cfg file itself, in which we only need to change one line – change the IP from 127.0.0.1 on 0.0.0.0 in field interface. We get the following.

{
	"settings": {
		"messages_capacity": 2048
	},
	"logging": [
		"info",
		"warn",
		"erro"
	],
	"language": "ENG",
	"address": {
		"interface": "0.0.0.0:9591",
		"incoming": "127.0.0.1:9592"
	},
	"connection": "127.0.0.1:9572"
}

Also, after editing the config, we simply restart the service: systemctl restart hidden_lake_messenger.service. We wait a little and check the work through systemctl status hidden_lake_messenger.service.

To check the operation of the messenger, we just need to find out the IP of the Orange Pi device. The router gave me IP=192.168.0.121. Therefore, we go to this IP by port – 9591. We get the following.

If we go through the Friends, Settings sections, we will receive approximately the following logs from the running HLM service.

Now, to test the functionality of HLM, we need to connect to someone on the Hidden Lake network. For this procedure it is necessary to exchange public keys. How this exchange will take place is up to the users. Either this is an offline exchange, or using a previously secure connection, or using certain protocols, such as this one. Hidden Lake is F2F networkTherefore, both subscribers must have public keys.

In my case, I simply transferred the public key from the Orange Pi to my computer, and from the computer I transferred another public key to the Orange Pi. In order to establish an F2F connection, you simply need to go to the section Friends add a friend and his public key.

And this is where the problems begin

In fact, problems with this whole idea could be noticed even at the time of installing and launching HLS, from the logs of which you can sometimes see WARN messages. This is not accidental, but let’s first look at this problem from a more applied level. As soon as I exchanged public keys, it was time to exchange messages in the messenger. I started writing the first messages from the HLM of the main computer and at the same time checking whether the messages were delivered. And only the third message was actually delivered to the HLM from the Orange Pi.

Here you should ask a completely logical question – how did this happen? But the answer is actually quite logical. Let’s start with the architecture of HLM itself. HLM contacts HLS to send and receive messages. In this case, HLM sets an additional option on its part, which tells HLS not to generate a response to the received message (in other words, not to ACK the fact that messages have been received). In a simpler analogy, this can be thought of as sending packets using UDP rather than TCP. The sender does not wait for a response indicating the successful receipt of messages from the recipient. He simply sends them to the network and hopes that the message will reach the sender. This architecture is actually not as strange as it might seem at first glance. This is due to the following conditions:

1. Accepting the answer slowed down the sending itself would be at least twice and N times if not received, where N is the ACK waiting constant,

2. Generating a response may partially lead to emergence of new types of de-anonymizing attacks on the part of trusted nodes, when they become capable of overloading the response queue, thereby leading to the possibility of tracking time and based on this finding the fact of communication with someone on the network,

3. Communication in HLS uses blind routing and the probability that the recipient, being on the network, will not receive a response will tend to zero as the number of nodes/relays in the network increases.

Okay, if we take all this into account, and especially the third point, then how did it happen that the message did not reach the recipient? And here we now need to look at another side of the issue – connections to the nodes themselves in the network. The network is designed in such a way that if a node starts sending a response, but does not send it to the end, then the nodes are forced to start from it switch off. The same situation occurs when nodes send information to another node, which in turn receives bytes very slowly. As a result, nodes also automatically terminate communication with the node. After an interruption, one of the nodes tries to re-establish the connection and this can happen indefinitely.

This can be observed in the HLS log when WARNs were present. Also, in addition to WARN, you should look at the time of receiving/sending messages. Ideally, each BRDCS should be generated in period equal to 5 seconds. Also, receiving encrypted messages from different nodes should be within 5 seconds. In other words, in the logs, when working correctly, there simply should not be a “doing nothing” interval of more than 5 seconds. But there are intervals were at 15 and 27 secondswhich is not good.

This tendency can also be easily tested through HLM itself. If we just constantly reload the Settings page, the connections will sometimes fail turquoise color, will sometimes become gray. Turquoise color is responsible for the successful existence of the connection, while gray indicates the absence of a real connection. In other words, the node is constantly disconnected (or is disconnected) and it constantly resumes connections.

Therefore, when the first messages were sent, the HLS node itself on the Orange Pi side was disconnected from the network. The third message arrived successfully, because HLS on the Orange Pi side was able to successfully connect to the network again for a certain period of time.

Okay, we figured that out, but the root cause of this behavior is still not clear, namely, why does the node decide to disconnect from the rest of the nodes in the network, or why do the remaining nodes in the network begin to disconnect from the node with the Orange Pi? And this is where the problem of low performance of the Orange Pi itself begins. it just doesn’t take the load off the network.

The network load consists of several resource-intensive operationsand also because of the network architecture itself, where an increase in the number of network participants linear affects the increase in load on the entire network. Resource-intensive operations consist of:

1. Proof of work. The HLS network sets the complexity to 20 bits for each message sent. The sending (or accumulation of false) messages from the node must occur every 5 seconds,

2. Encryption and signing. Each message sent must encrypt the session key with the recipient’s 4096-bit public key, and also sign the message hash with the sender’s 4096-bit private key.

3. Decodings. Each received message goes through a procedure of attempting to decrypt the message with a private key (the session key is decrypted) of 4096 bits. The node must be able to process N packets every 5 seconds, where N is the number of nodes in the network.

And then I remembered that earlier I tested a weak VPS with only three nodes, where one of them was the server itself. With this paradigm, the VPS managed to generate traffic itself and consume it. But during testing, when I created another node, the server began to lie down and get up in exactly the same way. This indicated that he had reached the peak of his load capacity. In such a paradigm, he could no longer effectively parallelize and manage to perform proof, encryption, signing and decryption operations in an interval of five seconds.

During the work, it is impossible to say for sure that the Orange Pi was loading the processor to a hundred percent, but sometimes this actually happened (perhaps when it successfully communicated with all the nodes). It felt like it was possible to fry micro fried eggs on the processor, but the temperature readings showed quite acceptable values.

Conclusion

During such an experimental phenomenon as rolling out an anonymous network on Orange Pi, both positive and negative results were obtained. In fact, the node started working, it connected to the network and could send and receive messages. But all this worked unstable, with at least four nodes on the network. If there were fewer nodes in the network, then the Orange Pi itself would behave more deterministically. In any case, it is certainly not advisable to use such devices with such a weak processor as a node in the Hidden Lake network. On the other hand, Orange Pi can easily be used as a relay in the Hidden Lake network, because relays do not generate traffic and do not try to decrypt it. As a result, the Orange Pi will have enough power for this type of task.

PS

Instead of using weak single-boards, like Orange Pi Lite, you can quite correctly use a mini PC, or the same single-boards, but with better characteristics (in terms of processor), as a Hidden Lake node. This should be enough to maintain normal operation with small groups of participants (10-30 nodes depending on the selected equipment).

For example, in one free time I tested Hidden Lake nodes on different VPS with three nodes placed on one server with the characteristics of a single-core processor of ~3.4 GHz. In total, there were 4 nodes on the network (3 nodes on the VPS, 1 node on my computer). The processor power on such a server remained in the range of 40-50%, and no disconnections or switching of nodes from the network were observed. Thus, the quality of the processor is one of the most important criteria for raising nodes in the Hidden Lake network.