Python is easy. Go is simple. Simple != easy

Python and Go have different properties, and therefore can complement each other.

There is a common misconception that simple And easy – It is the same. After all, if a tool is easy to use, then its internal workings should be easy to understand, right? And the opposite is also true, right? In fact, it’s just the opposite. While in spirit both concepts point to the same thing (the result from the outside seems easy), in practice such superficial ease is achieved by enormous complexity underneath the hood.

Let’s consider Python. It is known how low the barrier to entry into this language is; this is why Python is a favorite choice as a first programming language. In schools, universities, research institutes and numerous companies around the world, Python is preferred precisely because everyone can understand it, regardless of their level of education, and an academic background is usually not required here at all. To work with Python, you rarely have to resort to type theory or understand where, what and how exactly is stored in memory, in which threads this or that piece of code is executed, etc. Moreover, Python allows you to get started with some of the most extensive libraries for scientific computing and systems programming. When you can handle that kind of power, even one line of code demonstrates why Python has become one of the most popular programming languages ​​on the planet.

This is where the nuances begin – it turns out that the ease with which you can express anything in Python is not in vain. Under the hood, Python is a heavy interpreter, and even to execute a single line of code, many operations must occur within it. If you’ve ever heard that Python is a “slow” language, then know that this apparent “slowness” is associated with a number of decisions that the interpreter has to make at runtime. But, it seems to me, this is not even the main problem. The complexity of the entire Python runtime and its ecosystem, along with some arbitrary decisions made about package management in the language, are the reasons for the extreme fragility of this environment. Due to its fragility, incompatibilities and run-time failures often occur. It’s quite common for you to step away from working on a Python application for a while, then come back to it a few months later and find that the ecosystem has changed so much that your old application can no longer even run.

Of course, this is a gross, even oversimplification: today even children know that such problems are solved with the help of containers. Indeed, thanks to Docker and other similar tools, it is possible to permanently “fix” dependencies in a Python code base, so that the code base will run practically forever. But, in fact, this is simply shifting responsibility and dumping all the complexity onto the OS infrastructure level. It’s not the end of the world, but you can’t turn a blind eye to this problem and underestimate it.

From ease to simplicity

If we set out to solve the problems that exist in Python, we would end up with something like Rust – a language that is extremely productive, but notorious for its high barrier to entry. In my opinion, Rust is not at all easy to use and, moreover, it is not at all simple. While today Rust is on the very wave of hype, with all my experience (I have been programming for about 20 years, I took my first steps in C and C++) I cannot look at a Rust code snippet and confidently read it from the page.

About five years ago I discovered Go while I was working with a Python-based system. Despite the fact that I didn’t get comfortable with the Go syntax the first time, it immediately became clear to me how simple the ideas underlying it are. The Go language is designed to be easy to understand for anyone in an organization, from a junior fresh out of college to a senior engineering manager who has just glanced at the code. I will say more, despite the simplicity of the Go language, its syntax is updated very rarely. The last major change was generics, which were added in version 1.18, and then after a serious discussion that lasted a full decade. In most cases, if you look at Go code written even five days ago, even five years ago, the code will look very recognizable and should just work.

But simplicity requires discipline. At first glance, the Go language can seem restrictive and even slightly retrograde. Especially when you compare Go code to something as terse as list or dictionary inclusion in Python:

temperatures = [
    {"city": "City1", "temp": 19},
    {"city": "City2", "temp": 22},
    {"city": "City3", "temp": 21},
]

filtered_temps = {
    entry["city"]: entry["temp"] for entry in temperatures if entry["temp"] > 20
}

Writing the same code in Go requires a lot more keyboard tapping, but ideally it will have one less layer of abstraction than Python, which depends on its interpreter under the hood:

type CityTemperature struct {
    City      string
    Temp float64
}

// ...

temperatures := []CityTemperature{
    {"City1", 19},
    {"City2", 22},
    {"City3", 21},
}

filteredTemps := make(map[string]float64)
for _, ct := range temperatures {
    if ct.Temp > 20 {
        filteredTemps[ct.City] = ct.Temp
    }
}

While similar code can be written in Python, there is an unspoken rule in programming: if the language provides more easy (read, more concise, more elegant) option, then programmers will lean towards it. But “ease” is a subjective concept, and “simplicity” should be easy for anyone. When the same action can be performed in several ways, this leads to the development of different programming styles, and often many styles can be combined in the same code base.

Yes, Go can be called verbose and “boring,” but when working with it, it’s easy to add another plus: the Go compiler has to do very little work when building the executable file. Compiling and running applications in Go is often as fast as Python (where you still need to have an interpreter) or Java (where you need to run a virtual machine). Not surprisingly, the fastest executable is the native executable. It is not as fast as its C/C++ or Rust counterparts, but it is several times simpler at the source code level. I’m willing to ignore this small “disadvantage” of Go. And for starters: Go binaries are linked statically. This means that they can be assembled anywhere and run on the machine you need. There will be no dependencies associated with the runtime or libraries. For convenience, we still wrap our Go applications in Docker containers. However, these applications are miniature and consume only a fraction of the memory and CPU time compared to similar applications written in Python or Java.

How to combine Python and Go to your advantage

The most pragmatic solution we have come to in our work is to combine the best features lightness Python and you just Go. We think Python is a great testing ground for prototyping. This is where ideas are born, scientific hypotheses are accepted or rejected. Python is simply designed for data science and machine learning, and since we have to constantly deal with these areas in our work, it hardly makes sense to reinvent the wheel in any other languages. In addition, Python underlies the Django framework, whose motto is rapid application development (it is unlikely that there will be any equal in this, but to complete the picture, of course, I will mention Ruby on Rails and Phoenix for Elixir).

Let’s say that a project needs to get by with minimal user control, and data administration needs to be organized within the application (we have the majority of such projects). In this case, we are making a skeleton application in Django, since it has a built-in Admin, which is a fantastically useful thing. Once a crude prototype model in Django begins to resemble a product, we look at how much of that model can be rewritten in Go. Since a Django application already has a database structure defined and an understanding of what the data models look like, it’s fairly easy to write code in Go that takes over from Django. After several iterations, we achieve symbiosis, where both halves coexist peacefully on top of the same database, and communication between them is carried out through the simplest messaging system. Ultimately, the Django “shell” turns into an orchestrator – that is, it is responsible for administration and launches those tasks, which are then processed in the application area written in Go. The part written in Go is responsible for everything else, from client APIs and endpoints to business logic and processing jobs that appear on the machine interface.

This symbiosis has not failed us so far and, I hope, will not fail us in the future either. Someday I will write a post in which I will analyze the architecture outlined here in more detail.

Thank you for your attention!