Understanding Decorators in Python

What are decorators?

Decorators are wrappers around functions (or classes) in Python that change the way that function works. A decorator abstracts its own functionality. Decorator notation is generally the least invasive. A developer can write his code however he wants and use decorators only to extend the functionality. This all sounds very abstract, so let’s look at some examples.

In Python, decorators are used to “decorate” functions (or methods). Perhaps one of the most popular decorators is @property:

class Rectangle:
    def __init__(self, a, b):
        self.a = a
        self.b = b

    @property
    def area(self):
        return self.a * self.b

rect = Rectangle(5, 6)
print(rect.area)
# 30

As seen in the last line, you can access area our Rectangleas an attribute, i.e. you don’t need to call the method area. Instead, when accessing areaas an attribute (without ()), the method is called implicitly because of the decorator @property

How it works?

Write @property before the function definition is the same as writing

area = property(area). In other words: property is a function that takes another function as an argument and returns a third one. This is how decorators behave.

As a result, decorators change the behavior of the function they are applied to.

We write our decorators

retry decorator

Let’s write our own decorators according to this vague definition in order to understand how they work.

Let’s say we have a function that we want to retry if it fails. We need a function (decorator) that will call our function once or twice (depending on how the function ends the first time).

Given our initial decorator definition, we can write a simple decorator like this:

def retry(func):
    def _wrapper(*args, **kwargs):
        try:
            func(*args, **kwargs)
        except:
            time.sleep(1)
            func(*args, **kwargs)
    return _wrapper

@retry
def might_fail():
    print("might_fail")
    raise Exception

might_fail()

Retry – the name of our decorator, which takes any function as an argument (func). Inside the decorator, a new function is defined and returned (_wrapper). At first glance, the definition of one function within another may seem somewhat unusual. Syntactically, however, this is perfectly normal and has a certain advantage, because the function _wrapper only exists inside our decorator’s namespace retry.

Note that in the example we only decorated our function with @retry. After decorator @retry no parentheses. Thus, when calling the function might_fail() decorator @retry will be called with our function (might_fail) as the first argument.

As a result, we process three functions:

• retry

• _wrapper

• might_fail

Sometimes you want a decorator to take arguments. In our case, we can set the number of retries as a parameter. However, the decorator must accept our function as the first argument. Remember, we didn’t need to call the decorator when decorating a function with it, so we just wrote @retrybut not @retry().

• A decorator is nothing but a function (which takes another function as an argument)

• To use a decorator, you need to place it before the function definition without calling it

Therefore, we could write a fourth function that takes the parameter we want as a configuration and returns a function that is actually a decorator (which takes another function as an argument).

Let’s try like this:

def retry(max_retries):
    def retry_decorator(func):
        def _wrapper(*args, **kwargs):
            for _ in range(max_retries):
                try:
                    func(*args, **kwargs)
                except:
                    time.sleep(1)
        return _wrapper
    return retry_decorator


@retry(2)
def might_fail():
    print("might_fail")
    raise Exception


might_fail()

Let’s break it down into components:

• First we had a function retry;

• Retry takes an arbitrary argument (max_retries in our case) and returns a function;

• retry_decorator is the function returned retry and in fact our decorator;

• _wrapper works the same as before (now it just does the maximum number of retries).

To define our decorator:

• This time might_fail decorated with a function call, i.e. @retry(2);

• retry(2) causes retrywhich returns the decorator itself;

• might_fail will eventually be decorated retry_decoratorsince this function is the result of calling retry(2).

Timer Decorator

Here is another example of a useful decorator. Let’s write a decorator that will return the execution time of functions.

import functools
import time

def timer(func):
    @functools.wraps(func)
    def _wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        runtime = time.perf_counter() - start
        print(f"{func.__name__} took {runtime:.4f} secs")
        return result
    return _wrapper

@timer
def complex_calculation():
    """Some complex calculation."""
    time.sleep(0.5)
    return 42

print(complex_calculation())

Conclusion:

complex_calculation took 0.5041 secs
42

As we can see, the timer decorator executes the code before and after the function and works exactly the same as in the last example.

functools.wraps

You may have noticed that the function itself _wrapper decorated @functools.wraps. This fact does not change the logic or functionality of our timer decorator in any way. You might as well not use at all @functools.wraps.

However, since our decorator @timer could have been written as: complex_calculation = timer(complex_calculation)the decorator will definitely change our function complex_calculation. In particular, it changes some attributes of special methods:

• __module__

• __name__

• __qualname__

• __doc__

• __annotations__

Using @functools.wraps all of these attributes revert to their default values.

Without @functools.wraps:

print(complex_calculation.__module__)       # __main__
print(complex_calculation.__name__)         # wrapper_timer
print(complex_calculation.__qualname__)     # timer.<locals>.wrapper_timer
print(complex_calculation.__doc__)          # None
print(complex_calculation.__annotations__)  # {}

With @functools.wraps:

print(complex_calculation.__module__)       # __main__#
print(complex_calculation.__name__)         # complex_calculation
print(complex_calculation.__qualname__)     # complex_calculation
print(complex_calculation.__doc__)          # Some complex calculation.
print(complex_calculation.__annotations__)  # {} 

Class Decorators

So far, we’ve looked at function decorators. However, they can also be used for classes.

Let’s take our timer from the example above. We can quite easily wrap a class in it:

@timer
class MyClass:
    def complex_calculation(self):
        time.sleep(1)
        return 42

my_obj = MyClass()
my_obj.complex_calculation()

As a result:

Finished 'MyClass' in 0.0000 secs

It is obvious that the execution complex_calculation didn’t take time. Remember that the @ notation is just the equivalent of writing MyClass = timer(MyClass), i.e. the decorator will only be called when you “call” the class. Calling a class means creating an instance of it, so the timer will only work for the line my_obj = MyClass().

Class methods are not automatically decorated when the class is decorated. Simply put, when using a decorator for an ordinary class, only its constructor is decorated (method __init__).

You can change the behavior of the class as a whole by using a different form of the constructor. However, let’s first see if decorators can work the other way around, that is, if we can decorate a function with a class. It turns out we can:

class MyDecorator:
    def __init__(self, function):
        self.function = function
        self.counter = 0
    
    def __call__(self, *args, **kwargs):
        self.function(*args, **kwargs)
        self.counter+=1
        print(f"Called {self.counter} times")


@MyDecorator
def some_function():
    return 42


some_function()
some_function()
some_function()

Conclusion:

Called 1 times
Called 2 times
Called 3 times

How it works:

• __init__ called at checkout some_function. Again, remember that decorating is the same as some_function = MyDecorator(some_function).

• __call__ called when an instance of the class is used, such as when a function is called. Insofar as some_function is now an instance of my decorator, but we still want to use it as a function, we’ll need a special method __call__ .

Decorating a class in Python works like modifying a class from outside (i.e. from a decorator).

See:

def add_calc(target):

    def calc(self):
        return 42

    target.calc = calc
    return target

@add_calc
class MyClass:
    def __init__():
        print("MyClass __init__")

my_obj = MyClass()
print(my_obj.calc())

Conclusion:

MyClass __init__
42

Again, remember the definition of a decorator, because everything that happens here follows the same logic:

• my_obj = MyClass() first calls the decorator,

• Decorator add_calc adds a method calc to the class

• As a result, the class is created using the constructor.

You can use decorators to change classes like inheritance. Whether this is good or bad depends largely on the architecture of your project as a whole. Decorator dataclass from the standard library is a great example of sensible usage where decorators are preferred over inheritance. We’ll talk about it now.

Using Decorators

Decorators in the Python Standard Library

In the following sections, we’ll take a look at some of the popular and useful decorators that are already in the standard library.

property

As we already know, the decorator @propertyis probably one of the most used decorators in Python. It is needed so that you can access the result of the method as an attribute. Of course, there is an analogue @propertywith which you can call the method under the hood when performing an assignment operation.

class MyClass:
    def __init__(self, x):
        self.x = x
    
    @property
    def x_doubled(self):
        return self.x * 2
    
    @x_doubled.setter
    def x_doubled(self, x_doubled):
        self.x = x_doubled // 2

my_object = MyClass(5) 
print(my_object.x_doubled)  #  10  
print(my_object.x)          #  5  
my_object.x_doubled = 100   #    
print(my_object.x_doubled)  #  100 
print(my_object.x)          #  50  

staticmethod

Another popular decorator – staticmethod. It is needed if you want to call a function defined inside a class without creating an instance of the class:

class C:
    @staticmethod
    def the_static_method(arg1, arg2):
        return 42

print(C.the_static_method())

functools.cache

When you are dealing with functions that perform complex calculations, you may want to cache their result.

You can do something like this:

_cached_result = None
def complex_calculations():
    if _cached_result is None:
        _cached_result = something_complex()
    return _cached_result

Saving a global variable such as _cached_resultchecking it for None and putting the result into that variable are repetitive tasks. All this makes them an ideal candidate for the position of decorator. Luckily, the Python standard library has a decorator that does all this for us:

from functools import cache

@cache
def complex_calculations():
    return something_complex()

Now every time you call complex_calculations()Python first checks for a cached result before calling something_complex. If there is a result in the cache, something_complex will not be called twice.

Dataclasses

In the section on class decorators, we saw that they can be used to change the behavior of classes in a similar way to inheritance.

Module dataclasses in the standard library is a good example where using a decorator is preferable to using inheritance. Let’s first see how to use dataclasses:

from dataclasses import dataclass

@dataclass
class InventoryItem:
    name: str
    unit_price: float
    quantity: int = 0

    def total_cost(self) -> float:
        return self.unit_price * self.quantity


item = InventoryItem(name="", unit_price=12, quantity=100)
print(item.total_cost())    # 1200

At first glance, the decorator @dataclass only added a constructor, so we avoided boilerplate code like this:

...
    def __init__(self, name, unit_price, quantity):
        self.name = name
        self.unit_price = unit_price
        self.quantity = quantity
...

However, if you decide to write a REST API for your Python project and need to convert your Python objects to JSON strings, there is a package called dataclasses-json (not in the standard library), which decorates classes and ensures that objects are serialized and deserialized to JSON strings and vice versa.

Let’s see what it looks like:

from dataclasses import dataclass
from dataclasses_json import dataclass_json

@dataclass_json
@dataclass
class InventoryItem:
    name: str
    unit_price: float
    quantity: int = 0

    def total_cost(self) -> float:
        return self.unit_price * self.quantity


item = InventoryItem(name="", unit_price=12, quantity=100)

print(item.to_dict())
# {'name': '', 'unit_price': 12, 'quantity': 100}

From this we can draw two conclusions:

• Decorators can be nested. The order in which they are written is important.

• Decorator @dataclass_json added a method to our class to_dict

Of course, we could write a mixin class that does the hard work of implementing the method to_dictdata type safe and then inherit the class InventoryItem From him.

However, in this case, the decorator only adds technical functionality (as opposed to extending functionality). As a result, we can simply “turn on and off” the decorator without changing the behavior of our application. The “natural” class hierarchy is preserved, and no changes to the code need to be made. We can add a decorator dataclasses-json into the project without changing existing methods.

In this case, modifying a class with a decorator looks more elegant (because it maintains modularity) than inheritance or using mixins.

Tomorrow at OTUS there will be an open lesson “Docker for a Python developer”. Consider the best practices for writing Dockerfiles and working with docker in general. We will discuss the nuances of both a general nature and Python-specific ones. Registration – here.