The Lost Potential of Windows Subsystem for Linux (WSL)

If you have not followed Windows 10 at all for several years and do not know what is happening, then you missed one thing – a very hot topic for developers has become Windows subsystem for Linux, aka WSL. It is often discussed among programmers. Indeed, an amazingly interesting thing.

Finally, we have the opportunity to run our Linux toolkit on Windows along with Windows programs. This means that you no longer need to learn strange PowerShell or use an archaic console. CMD.EXE

Unfortunately, not everything is so rosy. WSL is still some kind of foreign element that is separate from the native Windows environment. In particular, it cannot interact with native Windows tools.

But initially everything was conceived quite differently, writes Julio Merino, Developer Blog jmmv.dev… The subsystem was supposed to be completely different, but in fact it was a failure, in a sense.

To understand the reasons for this failure, one must first understand the differences between WSL 1 and WSL 2 and how the move to WSL 2 closed some interesting prospects.

WSL Architecture Overview 1

Let’s take a look at WSL 1 first, and first of all at its odd name. Why is this feature called Windows Subsystem … for Linux? Isn’t it the other way around? This is not a subsystem in Linux that does something related to Windows, but the opposite! That is, it should be correctly called “Subsystem with Linux functionality for Windows” or “Linux Subsystem for Windows” or LSW. Where does the confusion come from?

there is opinionthat Microsoft was forced to choose the “reverse” name in order to avoid mentioning other people’s trademarks. Here are the words of Rich Turner, Project Manager for WSL:

Well … on the other hand, such a strange name is technically correct if you carefully examine the architecture of the Windows NT kernel. On the page “Windows NT Architecture” on Wikipedia we find the following:

“Windows NT user mode consists of subsystems that pass I / O requests to the appropriate kernel-mode driver through the I / O manager. There are two subsystems at the user level: environment subsystems (run applications written for different operating systems) and integrated or internal subsystems (manages specific system functions on behalf of the environment subsystem). Kernel mode has full access to the hardware and system resources of the computer. “

Windows NT was designed from the ground up to support processes running from a variety of operating systems, and Win32 was “just” one of these environmental subsystems. On such a platform, WSL 1 provides a new environment subsystem, the Linux subsystem, for running Linux binaries on top of the Windows NT kernel. The Win32 and Linux environment subsystems must have one common integral subsystem.

What does all this really mean?

Various systems are called “frontends” – that’s what that means. A user-space process is a collection of binary instructions that the processor executes continuously (interrupts aside). The operating system kernel is unaware of what the process is doing until the process issues a system call: at this point, the kernel regains control to perform an operation on behalf of the user, which can be something like reading a file or pausing for a few seconds.

The way in which a process makes system calls and the semantics of those system calls are operating system specific. For example, on old x86 it looks like this: opening a file in a Win32 system is a system call 17hwhich is initiated via interrupt INT 2EH, and when opening a file on Linux system, it is a system call 5hwhich is initiated via interrupt INT 80х

But … conceptually opening a file is opening a file, right? We are not particularly interested in the fact that system call numbers or interrupt numbers are different from each other. And therein lies a key design aspect of WSL 1: the Linux subsystem in the NT kernel is, simply put, the implementation of the Linux system call level before the NT kernel. These system calls are later delegated to NT primitives rather than Win32 calls. Most important: no no translation from Linux system calls to Win32 system calls

In a sense, this is a feat of architectural thought and real software development, given the generally good support for Linux applications under WSL 1 and keeping in mind the many real internal differences between NT and Unix, when Windows does not natively perceive standard Unix logic at all fork – exec

The true beauty of this design is that there is one core running on the machine, and that core has a holistic view of all the processes underneath. The kernel knows everything about Win32 and Linux processes. And all of these processes interact with shared resources such as a single networking stack, a single memory manager, and a single process scheduler.

Reasons for creating WSL 2

If WSL 1 is so cool, why do you need WSL 2? There are two reasons:

  • WSL 1 should, in fact, implement all the Linux kernel ABIs, as they say, bit by bit. If there is a bug in the interface, WSL 1 should reproduce it. And if there is a function that is difficult to represent in the NT kernel, then it either cannot be implemented, or it needs additional kernel logic (and therefore becomes slower).


    Levels and interfaces between them: API and ABI. High-level comparison

  • The Linux subsystem in WSL 1 must abide by any “constraints” and internal differences between the NT kernel and traditional Unix design. The most obvious difference is the NTFS file system and its semantics, and how these differences harm the performance of Linux binaries. Poor file system performance was apparently a common complaint when using WSL 1.

WSL 2 throws away all part of the Linux subsystem and replaces it with a complete (but very well hidden and fast) virtual machine. The virtual machine then runs the normal Linux kernel internally, the correct Linux filesystem, and the standard Linux networking stack. All this works inside the VM.

This means that the beauty of WSL 1’s design is gone: the Windows NT kernel no longer sees anything in the Linux world. Just a big black box that does something unknown within itself. Windows NT kernel only sees mount points VMENTER and VMEXIT for the virtual machine and block-level read / write requests on the virtual disk. This very NT kernel now knows nothing about Linux processes and file access. Likewise, a running Linux kernel knows nothing about NT.

Some other differences can be read in official documentation

Lost potential

From a user perspective, WSL 2 looks better: indeed, Linux applications are now much faster because they don’t go through the awkward “emulation” of Linux system calls in the NT kernel. If NTFS is difficult to use with Linux semantics, now there is no such problem, because now the Linux environment uses ext4 on its virtual disk. And support for Linux applications could be much more complete because … well, because WSL 2 is full Linux.

But think, how is this convenience achieved? What have we lost?

What WSL should have become if everything worked as intended:

  • Imagine how great it is to type ps or top in a WSL session and see Linux and Windows processes nearby, and any of them can be killed with the command kill?
  • Imagine how cool it is to manipulate Windows services from a WSL session?
  • How cool to use ifconfig (analogue of ipconfig from Windows, although there is still ip) within a WSL session to check and change the network interfaces of the computer?

  • In fact, you can imagine doing absolutely of all system administration tasks in Windows from linux console WSL? It’s a fairy tale!

Although this never existed, such a world is quite imaginable … and it could provide WSL 1 architecture only… And this is not fancy at all, because this is the model that macOS uses (although this is a bit cheating, because macOS is, in fact, Unix).

That’s what pisses me off the most writes Giulio Merino: “Although I can install WSL on my Azure development machine, I just can’t use it for anything at all. I still have to work in CMD.EXE, because it interacts with native Windows processes and resources, and the tools I deal with are only for Windows. “

On Q&A page it is written that WSL 1 will not be abandoned, that is, you can run WSL 1 and WSL 2 distributions together, upgrade any distribution to WSL 2 at any time, or return to WSL 1. And if you look at the strict adherence to Microsoft backward compatibility, due to which we still have to use archaic tools and protocols, this may be true. But maintaining WSL 1 is a colossal effort, because you have to track and comply with all Linux changes. Anyway, let’s hope that WSL 1 will continue to exist. And who knows, what if someday the magical world that we imagine will come true?

BSD compatibility level

Interestingly, the BSD family of operating systems (FreeBSD, OpenBSD, NetBSD, MidnightBSD, GhostBSD, Darwin, DragonFly BSD) have always lagged behind Linux and other commercial operating systems. But they had this binary compatibility that we are talking about a long time ago. Giulio Merino says that Linux compatibility in the NetBSD kernel was implemented back in 1995, that is, a quarter century ago and 21 years before WSL 1 was born.

Best of all, this compatibility isn’t limited to Linux. NetBSD has supported emulation of various operating systems over the years. SVR4 support appeared in 1994, and for a short period of time NetBSD even supported … PE / COFF binaries – yes, that’s right, these are Win32 binaries! So, in a sense, NetBSD implemented the WSL 1 model the other way around: it allowed Win32 binaries to run on top of the NetBSD kernel back in 2002. So that’s it.


Advertising

VDSina offers VDS for rent for any task, a huge selection of operating systems for automatic installation, it is possible to install any OS from your own ISO, a convenient control panel of our own design and a daily payment of the tariff, which you can create individually for your tasks.

Similar Posts

Leave a Reply