Asynchronous django: continued

I wrote a while ago about how you can add asynchrony to django. Now the last post is only of historical interest, if you haven’t read it, then don’t read it.

I wrote only about django orm, and the stumbling block there is accessing the database, but the approach can be generalized to any code containing I / O operations.

What we want to get: in general, native support for asynchrony, in any form, and in this article – so that you can use the entire django API in an asynchronous context without changing anything. For example, make calls await MyModel.objects.get()

In general, it turns out that adding asynchrony to django is essentially no more difficult than porting it to asynchronous once and for all. You can roughly imagine how it should be done: we take an asynchronous driver with an API roughly similar to django drivers and replace in the depths of the code calls like driver.execute_sql(sql, params) to calls like await async_driver.execute_sql(sql, params)… Of course, we will immediately receive SyntaxError because you cannot use await in a normal function, and we will have to add async and await to the entire call stack.

The entire django API is perfectly applicable to the asynchronous version, with a little adaptation. For example, there will be no more lazy attributes, and access to such an attribute will be possible if you previously fetched it from the database, for example, using prefetch_related or select_related (to access an attribute that requires accessing the database, you need a different API).

In principle, the work is simple and even relatively small. So, my point of view is that adding asynchrony (without breaking the old, synchronous, API) is just as easy – perhaps with a little overhead.

In general, there is such a thing in python as yield from, which is equally applicable in both synchronous and asynchronous contexts. Under the hood, of course, it doesn’t change anything. In my opinion, it should be used for projects that want to support both synchronous and asynchronous versions. Even for those who write from scratch – well, it seems to me. And for Legacy – no talk at all.

So we have pull request – to the main django branch (by the way, the pool from the main branch has never led to conflicts). It, of course, only supports a small portion of the django API – of course, because there were no other goals.

What do we see in it? Of course, a lot of yield from calls before functions – as I promised.

SyncDriver and AsyncDriver are not actually drivers, but classes with helper functions. It’s too lazy to rename, and it’s not very clear what to call. As you can see, they implement the function execute(operations)The fact is that we turned our code into a generator, and these same 2 classes, which are not drivers, request “operations” from it, one at a time, which need to be executed: functions, as they are, with parameters.

Where are the drivers themselves? This is also quite interesting. Here’s where:

class SQLCompiler:
    a = Asynchrony()

    @a.branch()
    def execute_sql(self, result_type=MULTI, chunked_fetch=False, chunk_size=GET_ITERATOR_CHUNK_SIZE):
        ...

    @a.branch()
    async def execute_sql(self, result_type=MULTI, chunked_fetch=False, chunk_size=GET_ITERATOR_CHUNK_SIZE):
        ...

As you understand, these are 2 branches of the same function – synchronous and asynchronous, which use the driver. It’s that simple, and you can have as many such branches as you want. By the way, asyncpg is not in the least compatible with dbapi – for no good reason, I’m sure.

Everything else is not that interesting. This is how, for example, I made Awaitable quersets

def __await__(self):
    return self.__iter__().__await__()

Or, for example, many of our functions contain yield from – what to do if our function is itself a generator (in django, yield is often used to create iterators). I found a simple solution: put the yield constructs inside another function (in my case – def iteratorand they won’t conflict).

You can also see the decorator @consume_generator… It turns the generator into a regular function (which will possibly return a coroutine). Designed to be called from outside (read by user). In general, the curious will see everything.

It’s all. The pwtail / django repository can be set with asterisks

Similar Posts

Leave a Reply