Clean Architecture through the eyes of a Python developer

Hello! My name is Eugene, I am a Python developer. Over the past year and a half, our team began to actively apply the principles of Clean Architecture, moving away from the classic MVC model. And today I will talk about how we came to this, what it gives us, and why the direct transfer of approaches from other PLs is not always a good solution.

Python has been my primary development tool for over seven years now. When they ask me what I like most about him, I reply that this is his excellent readability. The first acquaintance began with reading a book “Programming a collective mind”. I was interested in the algorithms described in it, but all the examples were in a language that was not yet familiar to me then. This was not usual (Python was not yet mainstream in machine learning), listings were often written in pseudo-code or using diagrams. But after a quick introduction to the language, I appreciated its conciseness: everything was easy and clear, nothing superfluous and distracting, only the very essence of the described process. The main merit of this is the amazing design of the language, the very intuitive syntactic sugar. This expressiveness has always been appreciated in the community. What is it worth?import this”, Always present on the front pages of any textbook: he looks like an invisible overseer, constantly evaluates your actions. On forums it was worth the noviop to use somehow Camelcase in the variable name in the listing, so immediately the discussion angle shifted towards the idiom of the proposed code with references to PEP8.

The desire for elegance plus the powerful dynamism of the language have allowed the creation of many libraries with a truly delightful API.

However, Python, although powerful, is just a tool that allows you to write expressive, self-documenting code, but does not guarantee thisas does not guarantee this and compliance with PEP8. When our seemingly simple online store on Django starts to make money and, as a result, pump up features, at one point we realize that it is not so simple, and even making basic changes requires more and more effort, and most importantly, this trend everything is growing. What happened and when everything went wrong?

Bad code

Bad code is not one that does not follow PEP8 or does not meet the requirements of cyclomatic complexity. Bad code is, first of all, uncontrolled dependencieswhich lead to the fact that a change in one place of the program leads to unpredictable changes in other parts. We are losing control over the code; expanding the functionality requires a detailed study of the project. Such code loses its flexibility and, as it were, resists changes, the program itself becomes “fragile.”

Clean architecture

The chosen architecture of the application should avoid this problem, and we are not the first to encounter this: there has been a discussion in the Java community about creating an optimal application design for a long time.

Back in the 2000th year Robert Martin (also known as Uncle Bob) in the article “Principles of Design and Design“Brought together five principles for designing OOP applications under the catchy acronym SOLID. These principles have been well received by the community and have gone far beyond the Java ecosystem. Nevertheless, they are very abstract in nature. Later there were several attempts to develop a general application design based on SOLID principles. These include: “Hexagonal architecture”, “Ports and adapters”, “Bulbous architecture” and they all have much in common, albeit different implementation details. And in 2012, an article was published by the same Robert Martin, where he proposed his own version called “Clean architecture“.

According to Uncle Bob, architecture is primarily “borders and barriers”, It is necessary to clearly understand the needs and limit the software interfaces in order not to lose control of the application. For this program divided into layers. Turning from one layer to another, you can transfer only data (simple structures and DTO objects) is a rule of borders. Another most frequently quoted phrase is that “the application should scream“- means that the main thing in the application is not the used framework or data storage technology, but what this application actually does, what function it performs – business logic applications. Therefore, the layers have not a linear structure, but have a hierarchy. Hence two more rules:

  • Inner layer priority rule – it is the inner layer that defines the interface through which it will interact with the outside world;
  • Dependency rule – Dependencies should be directed from the inner layer to the outer.

The last rule is quite atypical in the Python world. To apply any complicated business logic scenario, you always need to access external services (for example, a database), but to avoid this dependency, the business logic layer must declare the interface itselfby which he will interact with the outside world. This technique is called “dependency inversion“(The letter D in SOLID) and is widely distributed in languages ​​with static typing. According to Robert Martin, this OOP’s main advantage.

These three rules are the essence of Clean Architecture:

  • Border crossing rule;
  • Dependency rule;
  • The priority rule for the inner layer.

The advantages of this approach include:

  • Ease of testing – the layers are isolated, respectively, they can be tested without monkey-patching, you can granularly set the coating for different layers, depending on the degree of their importance;
  • Ease of changing business rules, since all of them are collected in one place, are not smeared according to the project and are not mixed with a low-level code;
  • Independence from external agents: the presence of abstractions between business logic and the outside world in certain cases allows you to change external sources without affecting the internal layers. It works if you have not tied the business logic to the specific features of external agents, for example, database transactions;
  • Perception Improvement, despite the fact that the code is spread out into layers, the high-level code does not mix with the low-level one.

Robert Martin considers his proposed four-layer scheme. In the framework of this article, I will not begin to disassemble it again. I redirect only those interested in the original article, and also to analysis on a habr I also recommend the excellent article “Clean Architecture Misconceptions.”

Python implementation

This is a theory, examples of practical application can be found in the original article, reports and book of Robert Martin. They rely on several common design patterns from the Java world: Adapter, Gateway, Interactor, Fasade, Repository, DTO and etc.

Well, what about Python? As I said, laconicism is valued in the Python community. What has taken root in others is far from the fact that it will take root with us. The first time I turned to this topic three years ago, then there were not many materials on the topic of using Clean Architecture in Python, but the first link in Google was project Leonardo Giordani: the author describes in detail the process of creating an API for a TDD-based real estate search site using Clean Architecture.
Unfortunately, despite the scrupulous explanation and following all the canons of Uncle Bob, this example is more likely scares off.

The project API consists of one method – obtaining a list with an available filter. I think that even for a novice developer, the code for such a project will take no more than 15 lines. But in this case, he took six packets. You can refer to a not entirely successful layout, and this is true, but in any case, it’s difficult for someone to explain the effectiveness of this approachreferring to this project.

There is a more serious problem, if you do not read the article and immediately begin to study the project, then it is quite difficult to understand. Consider the implementation of business logic:

from rentomatic.response_objects import response_objects as res

class RoomListUseCase(object):
   def __init__(self, repo):
       self.repo = repo
   def execute(self, request_object):
       if not request_object:
           return res.ResponseFailure.build_from_invalid_request_object(
           rooms = self.repo.list(filters=request_object.filters)
           return res.ResponseSuccess(rooms)
       except Exception as exc:
           return res.ResponseFailure.build_system_error(
               "{}: {}".format(exc.__class__.__name__, "{}".format(exc)))

The RoomListUseCase class that implements business logic (not very similar to business logic, right?) Of the project is initialized with an object repo. But what is repo? Of course, from the context we can understand that repo implements the Repository template for accessing data, if we look at the body of RoomListUseCase, then we understand that it must have one list method, the input of which is a list of filters, which is not clear at the output, you need to look in ResponseSuccess. And if the scenario is more complex, with multiple access to the data source? It turns out, in order to understand what repo is, you can just referring to the implementation. But where is she located? It lies in a separate module, which is in no way associated with RoomListUseCase. Thus, in order to understand what is happening, you need to go up to the upper level (the level of the framework) and see what is fed to the input of the class when creating the object.

You might think that I list the disadvantages of dynamic typing, but this is not entirely true. Exactly dynamic typing allows you to write expressive and compact code. The analogy with microservices comes to mind, when we cut a monolith into microservices, the design takes on additional rigidity, since anything can be done inside the microservice (PL, frameworks, architecture), but it must comply with the declared interface. So here: when we divided our project into layers, connections between layers must comply with the contractwhile a contract is optional inside the layer. Otherwise, you need to keep a fairly large context in mind. Remember, I said that the problem with bad code is dependencies, and so, without an explicit interface, we again slide back to where we wanted to get away – to lack of explicit causal relationships.

In this example, the interface repo is an integral part of the RoomListUseCase interface, like the execute method – this is how dependency inversion works. In fact, we can distribute the package with business logic separately from the application itself, since it does not have dependencies inside the project. When we work with a layer of business logic, we are not required to remember the existence of other layers. But in order to use it, it is necessary to implement the necessary interfaces, and repo one of them.

In general, at that time I abandoned Clean Architecture in a new project, again applying the classic MVC. But, having filled the next batch of cones, he returned to this idea a year later, when, at last, we began to launch services in Python 3.5+. As you know, he brought type annotations and data classes: Two powerful interface description tools. Based on them, I sketched a prototype of the service, and the result was already much better: the layers stopped scattering, despite the fact that there was still a lot of code, especially when integrating with the framework. But this was enough to start applying this approach in small projects. Gradually, frameworks began to appear that focused on the maximum use of type annotations: apistar (now starlette), moltenframework. The pydantic / FastAPI bundle is now common, and integration with such frameworks has become much easier. This is what the above restomatic / example would look like:

from typing import Optional, List
from pydantic import BaseModel

class Room(BaseModel):
   code: str
   size: int
   price: int
   latitude: float
   longitude: float

class RoomFilter(BaseModel):
   code: Optional[str] = None
   price_min: Optional[int] = None
   price_max: Optional[int] = None

class RoomStorage:
   def get_rooms(self, filters: RoomFilter) -> List[Room]: ...

class RoomListUseCase:
   def __init__(self, repo: RoomStorage):
       self.repo = repo
   def show_rooms(self, filters: RoomFilter) -> List[Room]:
       rooms = self.repo.get_rooms(filters=filters)
       return rooms

RoomListUseCase – a class that implements the business logic of the project. Do not pay attention to the fact that all that the show_rooms method does is call to RoomStorage (I did not come up with this example). In real life, there can also be a discount calculation, ranking a list based on advertisements, etc. However, the module is self-sufficient. If we want to use this scenario in another project, we will have to implement RoomStorage. And what is needed for this is clearly visible right from the module. Unlike the previous example, such a layer is self-sufficient, and when changing, it’s not necessary to keep the whole context in mind. From non-systemic dependencies only pydantic, why, it will become clear in the module for connecting the framework. No dependenciesAnother way to increase the readability of the code, not an additional context, even a novice developer will be able to understand what this module does.

A business logic scenario does not have to be a class; below is an example of a similar scenario in the form of a function:

def rool_list_use_case(filters: RoomFilter, repo: RoomStorage) -> List[Room]:
   rooms = repo.get_rooms(filters=filters)
   return rooms

And here is the connection to the framework:

from typing import List
from fastapi import FastAPI, Depends
from rentomatic import services, adapters
app = FastAPI()

def get_use_case() -> services.RoomListUseCase:
   return services.RoomListUseCase(adapters.MemoryStorage())"/rooms", response_model=List[services.Room])
def rooms(filters: services.RoomFilter, use_case=Depends(get_use_case)):
   return use_case.show_rooms(filters)

Using the get_use_case function, FastAPI implements a pattern Dependency injection. We do not need to worry about data serialization, all the work is done by FastAPI in conjunction with pydantic. Unfortunately, the business logic data format is not always suitable for direct broadcast to rest business logic should not know where the data came from – from url, request body, cookie, etc.. In this case, the body of the room function will have a certain conversion of input and output data, but in most cases, if we work with the API, such an easy proxy function is enough.

Here link to the example repository, where, in addition to the above modules, contains an elementary implementation of RoomStorage. In general, this is still more than 15 lines, which I spoke about earlier, but in this case it is justified, since the foundations for the further growth of the code base are laid.

I intentionally did not separate the layer of business logic, as the canonical Clean Architecture model suggests. The Room class was supposed to be in the Entity domain region layer representing the domain region, but for this example there is no need for this. From combining Entity and UseCase layers, the project does not cease to be a Clean Architecture implementation. Robert Martin himself has repeatedly said that the number of layers may vary both up and down. At the same time, the project meets the main criteria of Clean Architecture:

  • Border crossing rule: borders cross pydantic models, which are essentially DTO;
  • Dependency rule: the business logic layer is independent of other layers;
  • Inner layer priority rule: it is the business logic layer that defines the interface (RoomStorage) through which the business logic interacts with the outside world.

Today, several projects of our team, implemented using the described approach, are working on the prod. I try to organize even the smallest services in this way. It trains well – questions come up that I had not thought about before. For example, what is business logic here? This is far from always obvious, for example, if you are writing some kind of proxy. Another important point is learn to think differently. When we get a task, we usually start thinking about the frameworks, the services used, about whether there will be a need for a queue, where it is better to store this data, which can be cached. In the approach dictating Clean Architecture, we must first implement the business logic and only then move on to implementing interaction with the infrastructure, since, according to Robert Martin, the main task of architecture – delay as far as possible the moment when communication with any infrastructure layer will be an integral part of your application.

In general, I see a favorable prospect for using Clean Architecture in Python. But the form, most likely, will be significantly different from how it is accepted in other PLs. Over the past few years, I have seen a significant increase in interest in the topic of architecture in the community. So, at the last PyCon there were several reports on the use of DDD, and the guys from dry labs. In our company, many teams are already implementing the approach described to one degree or another. We are all doing the same thing, we have grown, our projects have grown, the Python community has to work with this, to determine the common style and language that, for example, once became for all Django.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *