YouTube slowdown from a technical perspective
I'll say right away that I'll talk about everything that is known at the moment. It's clear that I'm not the only one who's figured this out: a huge thank you to the regulars of the ntc party forum for the research they've done.
So, let's start from the beginning, with the ill-fated GGC. The official statement was that the server cache was outdated and had not been updated for a long time, but many had this skeptical feeling: “they worked before, and then everything broke at one moment.” In addition, there is Wikipedia articlewhich claims that talk about these GGCs has been going on since 2018. And if it were them, the lion's share of Google services would not work well for us, including Google Drive, Google PlayMarket, Google Images. But it works perfectly, as before.
Next, we will consider the idea that Google is testing the limitation of downloaders like yt-dlp, because, first of all, the problems affected them. The idea is actually interesting, because everything really works well on modern protocols (QUIC), and there are problems only on good old TCP (now there are rumors that QUIC has also begun to be limited, and, it seems, I can confirm this). We will refute this and the previous idea further.
As soon as I started to deal with this problem and opened Wireshark, I was waiting for logs consisting half of red stripes. I did not think much about how exactly YouTube slows down from the TCP side, but it immediately became clear that the slowdown is artificial, because if it were GGC, they either would not work at all, or the speed would simply be low, and here there are tons of lost packets. And it's also funny that with QUIC, which works automatically in Chrome, this did not happen at all and the speed was normal. I started searching and remembered one very cool forum for bypassing all sorts of similar jokes with Internet censorship, ntc party. There I found, how exactly YouTube is slowing down and a counter-example confirming 100% that this is the work of Big Brother.
But before we name the method of slowing down, let's think about how they can even determine that I'm watching YouTube just by seeing raw IP packets. Let's start with the lowest level that might interest them, IP. What's there? IP address. And this is a very powerful tool, though if there aren't thousands of them, and if only one service works on them, GoogleVideo. In the case of Google's huge infrastructure, it's simply difficult to get them, not to mention the fact that if, say, GoogleVideo and Google PlayMarket work on the same IP, and there is such a possibility, it's GGC after all. Then by cutting one, we'll cut the other too. What to do? Let's look higher. There's little interesting on TCP, let's go even higher. And then TLS. And that's it? Almost… Yes, modern protocols are designed to encrypt everything they can. But there is only one exception. The TLS specification has such a thing, as SNI. It is transmitted to the server and tells it which site we want to connect to and, therefore, which certificate to give us. Something like Host in HTTP, but for TLS. So, this very SNI is quietly transmitted to itself in unencrypted form. There were many specifications suggesting to encrypt it, but it did not become mainstream. In addition, providers blocked such connections so as not to interfere with monitoring users.
So, SNI is in unencrypted form in Client Hello, and having found it, we can mark the connection with a connmark and throw out random packets.
I know you're waiting for proof, gentlemen, and here it is:
curl --connect-to ::speedtest.selectel.ru https://manifest.googlevideo.com/100MB -k -o/dev/null What do we see in this curl request? Curl has the ability to connect to a server with its name, but instead of resolving DNS, put some of its own IP. In this case, all the information of a higher level will be transmitted as if we were using this name. This behavior makes curl replace that same SNI (well, and also Host, but we are not particularly interested in this). The -k flag is used to disable certificate checking (it will fail, because speedtest.selectel.ru will give us its certificate, but not the desired one from Google Video). And if we run this request, we will get a cheerful speed of 120 kilobytes at the beginning, and then it will go down. It can even go to zero. And now replace any letter in GoogleVideo with another and the speed will jump to normal Internet speed. Everything fell into place.
How to deal with this?
Of course, there are ways to combat this. For this, we will think in the direction of mangling (changing) packages. But if we simply replace googlevideo.com with yandex.ru, we will get a certificate error. And it is not a fact that the browser will allow you to disable this check for Google Video and will live with it normally. So what to do?
Remember that TSPU is a very highly loaded thing, which is unlikely to remember what and when we sent to the network, and, accordingly, if we simply split our request into two in the middle of SNI, then Oh Miracle! Everything starts working. In my case, I didn't even need fake Client Hello or a delay between packets (but it seems like it still needs the reverse order of packets: first the tail, then the head).
How to split? There are two options: IP fragments and at the TCP segmentation level. Personally, IP fragmentation worked half-heartedly for me at first, and then it completely failed. Perhaps the provider didn't like it. Let's go higher, to TCP. And here everything works very nicely for them. Even despite the fact that I seemingly made two packets out of one, changed the sizes of two packets, I did not violate any TCP connection rules: Ack by the number of bytes delivered is a thing.
Ready-made bypass tools. Well, here's the most delicious part. I personally wrote your solution for Linuxwhich is aimed only at YouTube. There is also a Windows version GoodbyeDPI by VladikSSthere is also one for Linux prohibit. Exists ByeDPIwhich works as a proxy (Windows/Linux). There is also ByeDPI version for Androidworks as a “fake vpn”. I recommend reading detailed comment from VladikSS on how to use these tools. If you want to dive deeper into this topic, you can take a closer look here: https://ntc.party/t/slowdown-youtube-in-russia/8055/ and https://ntc.party/t/discussion-slowdown-youtube-in-russia/8074/
