How we broke BattlEye packet encryption
Recently, Battlestate Games, the developers of Escape From Tarkov, hired BattlEye to implement encryption of network packets to prevent fraudsters from intercepting these packets, disassembling them and using them to their advantage in the form of radar cheats or otherwise. Today we will tell you in detail how we broke their encryption after a few hours.
We started by analyzing the “Escape from Tarkov” itself. The game uses the Unity Engine, which in turn uses C #, an intermediate language, which means it is very easy to view the game’s source code by opening it in tools like ILDasm or dnSpy. For this analysis, we worked with dnSpy.
Unity Engine without the IL2CPP option generates game files and places them in GAME_NAME_Data Managed, in our case it is EscapeFromTarkov_Data Managed. This folder contains all the dependencies using the engine, including the file with the game code – Assembly-CSharp.dll, we loaded this file into dnSpy, and then looked for the encryption string and ended up here:
This segment is in the EFT.ChannelCombined class, which, as you can tell from the arguments passed to it, works with the network:
Right clicking on the channelCombined.bool_2 variable, which registers as an indicator of whether encryption has been enabled, and then clicking on the Analyze button shows us that there are two methods referencing this variable:
The second one is the one we’re in right now, so double-clicking on the first brings us here:
Voila! There is a call to BEClient.EncryptPacket, clicking on the method will lead to the BEClient class, we can dissect it and find the DecryptServerPacket method. This method calls the pfnDecryptServerPacket function in the BEClient_x64.dll library. It will decrypt the data in the user’s buffer and write the size of the decrypted buffer to the pointer provided by the calling method.
pfnDecryptServerPacket is not exported by BattlEye and is not computed by EFT, in fact it is supplied by the BattlEye initializer that is called by the game at some point. We managed to calculate the RVA (Relative Virtual Address) by loading BattlEye into our process and copying how the game initializes it. The code for this program is here…
In the last section, we concluded that EFT calls BattlEye to complete all of its cryptographic tasks. So now we are talking about reverse engineering not IL, but native code, which is much more complicated.
BattlEye uses a security mechanism called VMProtect that virtualizes and modifies the segments specified by the developer. To properly reverse engineer a binary protected by this obfuscator, you need to unpack it.
Unpacking is a dump of a process image at runtime; we made a dump by loading it into a local process and then working in Scyllato flush its memory to disk.
Opening this file in IDA and then going to the DecryptServerPacket procedure will lead us to a function that looks like this:
This is called vmentry, it adds a vmkey to the stack and then calls the virtual machine handler vminit. The trick is this: because the instructions are “virtualized” by VMProtect, they are understandable only to the program itself.
Lucky for us, member of the Secret Club can1357 made a tool that completely breaks this protection – VTIL; you will find him here…
Finding out the algorithm
The generated VTIL file reduced the function from 12195 instructions to 265, which greatly simplified the project. Some VMProtect procedures were present in the disassembled code, but they are easily recognized and can be ignored, encryption starts from here:
Here is the pseudo-C equivalent:
uint32_t flag_check = *(uint32_t*)(image_base + 0x4f8ac); if (flag_check != 0x1b) goto 0x20e445; else goto 0x20e52b;
VTIL uses its own set of instructions to simplify the code even further. I translated it into pseudo-C.
We parse this routine by going into 0x20e445 which is a jump to 0x1a0a4a, at the very beginning of this function they move sr12 – a copy of rcx (the first argument in the default x64 calling convention) – and store it on the stack in [rsp+0x68]and the xor key is in [rsp+0x58]… Then this routine goes to 0x1196fd, here it is:
And here is the pseudo-C equivalent:
uint32_t xor_key_1 = *(uint32_t*)(packet_data + 3) ^ xor_key; (void(*)(uint8_t*, size_t, uint32_t))(0x3dccb7)(packet_data, packet_len, xor_key_1);
Note that rsi is rcx and sr47 is a copy of rdx. Since it is x64, they call 0x3dccb7 with arguments in this order: (rcx, rdx, r8). Luckily for us, vxcallq in VTIL means calling a function, pausing virtual execution, and then returning to the virtual machine, so 0x3dccb7 is not a virtualized function! By entering this function in IDA and pressing F5, you will invoke the pseudocode generated by the decompiler:
This code looks incomprehensible, there are some random assembler inserts in it, and they do not matter at all. Once we undo these instructions, change some var types, and then press F5 again, the code looks much better:
This function decrypts the packet into non-contiguous 4-byte blocks, starting at the 8th byte, using the rolling xor cipher key.
Note from the translator:
Rolling xor – a cipher in which the xor operation is literally rolled [отсюда rolling] by bytes:
The first byte remains unchanged.
The second byte is the xor result of the first and second original bytes.
The third byte is the XOR result of the modified second and original third bytes, and so on. Implementation here…
Continuing to look at the assembler, we will realize that it is calling another procedure here:
The x64 assembly equivalent:
mov t225, dword ptr [rsi+0x3] mov t231, byte ptr [rbx] add t231, 0xff ; uhoh, overflow ; the following is psuedo mov [$flags], t231 u< rbx:8 not t231 movsx t230, t231 mov [$flags+6], t230 == 0 mov [$flags+7], t230 < 0 movsx t234, rbx mov [$flags+11], t234 < 0 mov t236, t234 < 1 mov t235, [$flags+11] != t236 and [$flags+11], t235 mov rdx, sr46 ; sr46=rdx mov r9, r8 sbb eax, eax ; this will result in the CF (carry flag) being written to EAX mov r8, t225 mov t244, rax and t244, 0x11 ; the value of t244 will be determined by the sbb from above, it'll be either -1 or 0 shr r8, t244 ; if the value of this shift is a 0, that means nothing will happen to the data, otherwise it'll shift it to the right by 0x11 mov rcx, rsi mov [rsp+0x20], r9 mov [rsp+0x28], [rsp+0x68] call 0x3dce60
Before we continue to analyze the function it calls, we must come to the following conclusion: the shift is meaningless because the carry flag is not set, and this leads to the return value of 0 from the sbb instruction; in turn, this means that we are not on the right track.
If we look for references to the first procedure 0x1196fd, we will see that it is indeed referenced again, this time with a different key!
This means that the first clue was in fact pointing in the wrong direction, and the second is most likely the correct one. Good Bastian!
Now that we have dealt with the real key xor and the arguments to 0x3dce60, which are in this order: (rcx, rdx, r8, r9, rsp + 0x20, rsp + 0x28). Go to this function in IDA, press F5 – and now it’s very easy to read it:
We know the order of the arguments, their type and value; the only thing left is to translate our knowledge into real code, which we wrote well and wrapped in this gist…
This encryption was not the most difficult to reverse engineer, and our efforts were certainly noticed by BattlEye; after 3 days the encryption was changed to a TLS-like model where RSA is used for secure AES key exchange. This makes MITM without reading process memory impracticable in every sense and purpose.
If the field of information security is close to you, then you can turn your attention to our special course Ethical hacker, where we teach students to look for vulnerabilities even in the most reliable systems and make money on it.
find outhow to level up in other specialties or master them from scratch:
Other professions and courses