Installing and configuring tor in Linux Mint
Let's assume that you wanted to get information from a website located on the onion domain. In general, there is nothing special about this. Even, to some extent, easier than in clearnet. This is because very few scripts are used here. And therefore, you don’t have to resort to special tricks for parsing pages unless necessary. You just need to get there. And with a simple requests It won't be possible to do this. After some digging on the Internet, I found a solution that works for now.
Disclaimer: This information is taken from open sources and is intended solely to familiarize yourself with the operation of this technology. The author is not responsible for the use of this information for purposes other than information.
Step 1: Install tor
Start over . One by one, enter commands in the terminal to update packages and install the tor package:
sudo apt-get update
sudo apt-get install torAnd this is the simplest thing. Let's move on. The fact is that since 2022, Tor traffic began to be blocked. And just like that, without using bridges, you won’t be able to connect to the onion network. Therefore, to use bridges, you need to install the following package:
sudo apt install obfs4proxyAfter this we get a list of bridges. Let's go to this one page and we carry out step by step everything that is asked of us until we get the desired result. Well, or you can make it simpler. Find and launch in telegram bot and ask him for bridges. Then open the tor configuration file:
sudo xed /etc/tor/torrcI did it in the least painless way possible. After the configuration opens, you need to add the following lines to the end of the file:
ClientTransportPlugin obfs4 exec /usr/bin/obfs4proxy
Bridge obfs4 185.246.188.76:7180 0F14...
Bridge obfs4 208.87.97.172:1701 08AA...
Bridge obfs4 54.196.245.125:9162 7966...
UseBridges 1 Here you should pay attention to the fact that to indicate a bridge, we first write the keyword: Bridge, and only then the bridge. Save the configuration and restart tor.
sudo /etc/init.d/tor restartYou can check if the service is running using the command:
/etc/init.d/tor statusYou will get something like this in response. Well, or they should get it.
● tor.service - Anonymizing overlay network for TCP (multi-instance-master)
Loaded: loaded (/lib/systemd/system/tor.service; disabled; vendor preset: enabled)
Active: active (exited) since Sat 2024-03-02 03:07:00 +06; 35min ago
Process: 16751 ExecStart=/bin/true (code=exited, status=0/SUCCESS)
Main PID: 16751 (code=exited, status=0/SUCCESS)
CPU: 2ms
мар 02 03:07:00 hm systemd[1]: Starting Anonymizing overlay network for TCP (multi-instance-master)...
мар 02 03:07:00 hm systemd[1]: Finished Anonymizing overlay network for TCP (multi-instance-master).
Hint: Some lines were ellipsized, use -l to show in full.To stop and start tor you need to use the following commands:
sudo /etc/init.d/tor start
sudo /etc/init.d/tor stopAnother caveat is that if you do not want tor to start with the system, you should run the command:
sudo systemctl disable tor.serviceAnd only now can you move on to the second step.
Step 2: Install Python Libraries
In order not to drag out the story, we will install the necessary libraries using one command:
pip install requests requests[socks] requests[security]Now that everything we need is installed and configured, we can move on to writing code. There's nothing special here. With the exception of some nuances, which I will discuss below. In general, after installing libraries that allow you to use a proxy, the code is no different from normal.
import requests
proxies = {
'http': 'socks5h://127.0.0.1:9050',
'https': 'socks5h://127.0.0.1:9050'
}
url="http://flibustaongezhld6dibs2dps6vm4nvqg2kp7vgowbu76tzopgnhazqd.onion/"
r = requests.get(url, proxies=proxies, verify=False) # using TOR network
print(r.text)There is a nuance in specifying a proxy. If you specify socks5://127.0.0.1:9050, you may not get anything done. At least nothing happened to me. From the word absolutely. And only after I added the letter h to make socks5h://127.0.0.1:9050, everything fell into place and the code worked as usual.
In the code above, I get the html code of the main page of the Flibusta library, which, as you know, has long been firmly established in onion. But that’s not about that now. Let's try to execute the code and get the headers as well.
In the end, this is what we got:
{'Server': 'nginx', 'Date': 'Fri, 01 Mar 2024 21:49:36 GMT', 'Content-Type': 'text/html; charset=utf-8', 'Transfer-Encoding': 'chunked',
'Connection': 'keep-alive', 'Keep-Alive': 'timeout=10', 'Vary': 'Accept-Encoding, Cookie', 'Cache-Control': 'public, max-age=600',
'Last-Modified': 'Fri, 01 Mar 2024 21:49:36 +0000', 'Expires': 'Sun, 11 Mar 1984 12:00:00 GMT', 'ETag': 'W/"1709329776"',
'Content-Encoding': 'gzip'}Above are the headers, below is the page code. Of course not all. Just the beginning.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" lang="ru" xml:lang="ru">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Флибуста | Книжное братство</title>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<link href="https://habr.com/opds" type="application/atom+xml;profile=opds-catalog" rel="related" />
<link rel="alternate" type="application/rss+xml" title="Флибуста RSS" href="http://flibustaongezhld6dibs2dps6vm4nvqg2kp7vgowbu76tzopgnhazqd.onion/rss.xml" />It turned out to be a sort of spontaneous instruction. Why is this even needed? There can be many applications here. Including parsing sites on the regular network not from your address. This does not work with all sites. Some block access from Tor addresses. However, you can try. There are instructions on how to write code that will automatically change identities after several iterations.
Well, the most banal thing is why I need this – just not to forget. Sometimes you need to find something. And you remember seeing this somewhere. Just where… It takes quite a lot of time to search. And so, the summary instructions will be available in one place. At least for me. Well, for those who have it.
And that's all. Thank you for your attention. I hope that this information will be useful to you.
Subscribe to our telegram channelwe have a lot of useful things!
