Once again about the asynchronous state machine and where exactly are the allocations

Despite the fact that about async/await Many words have already been said and many reports have been recorded, however, in my practice of teaching and mentoring, I often encounter misunderstandings of the device async/await even among Middle+ level developers.

As you know, when compiling an asynchronous method, the compiler transforms the code, breaking it into separate steps. Then, during execution, each step is interrupted by an asynchronous operation. When it ends, you need to understand exactly where to return control – to which specific step. Therefore, all steps are numbered and the compiler very strictly monitors where you can go from where to where. In computer science this solution is called state machine. Also, in Russian they call it state machine. In what follows, for brevity, I will use the abbreviation SM (state machine).

So, in this article we will take a closer look state machinegenerated by the C# compiler from an async method to understand how async works in C#.

“High-level” C#

First, let’s look at an example of simple code in regular (“high-level”) C#.

using System;
using System.Threading.Tasks;
using System.IO;

public class Program {
    private string _fileContent;
    
    public async Task Main() {
        await Task.Delay(100);
        
        int delay = int.Parse(Console.ReadLine());
        await Task.Delay(delay);
        
        _fileContent = await File.ReadAllTextAsync("file1");
        
        await Task.Delay(delay);
    }
}

The code first waits 100 ms, then reads from the console how much more to wait, waits more, reads the data from the file, and waits some more. There is no need to look for logic in the sequence of these calls; the main thing for us here is that these are simply understandable asynchronous calls.

“Low-level” C#

The following is the code that the compiler generates from “high-level” (regular) C#.
I’ll say right away that the original code generated by the compiler looks as if the developers of the code generator did everything to ensure that nothing was incomprehensible to a person. However, for those interested, the original code can be found look at sharplab.io.

In general, I asked The neural network carried out a small refactoring to make the code more readable, and also accompanied significant and non-obvious places with detailed comments. This is what happened (class contents Program):

// Машина состояний (SM)
private sealed class AsyncStateMachine : IAsyncStateMachine
{
    // Определение состояний для машины состояний
    public enum State
    {
        NotStarted,               // Машина состояний не запущена - начальное состояние
        WaitingAfterInitialDelay, // Ожидание после начальной задержки
        WaitingForFileRead,       // Ожидание чтения файла
        WaitingAfterFinalDelay,   // Ожидание после последней задержки
        Finished                  // Завершено
    }

    public State CurrentState; // Текущее состояние машины состояний

    public AsyncTaskMethodBuilder Builder; // Строитель задачи асинхронного метода

    public Program Instance; // Экземпляр программы (оригинального класса)

    private int DelayDuration; // Длительность задержки (переменная delay стала полем машины состояний)

    private string FileContentTemp; // Временное хранение содержимого файла

    private TaskAwaiter DelayAwaiter; // Ожидатель задержки

    private TaskAwaiter<string> ReadFileAwaiter; // Ожидатель чтения файла

    private void MoveNext()
    {
        try
        {
            switch (CurrentState)
            {
                case State.NotStarted:
                    // Запуск начальной задержки
                    DelayAwaiter = Task.Delay(100).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        /* 
                            В случае если таска сразу после запуска завершилась, произойдет переход к выполнению следующего этапа машины состояний (WaitingAfterInitialDelay)
                            Такое бывает, например, когда в методе с модификатором async нет асинхронных вызовов, либо если мы эвэйтим уже завершенную таску.
                        */
                        goto case State.WaitingAfterInitialDelay;
                    }
                    // Конкретно в этом кейсе, исполнение не зайдет в if, который выше, а выполнит две нижние строки
                    CurrentState = State.WaitingAfterInitialDelay;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    /* 
                        AwaitUnsafeOnCompleted запланирует, что указанная машина состояний (ref this) будет продвинута вперед после завершения работы указанного awaiter'а (будет вызван метод MoveNext).
                        По смыслу это похоже на ContinueWith.
                        [ссылка на исходник под кодом] *
                    */
                    break;

                case State.WaitingAfterInitialDelay:
                    DelayAwaiter.GetResult();
                    /*
                        В случае если в асинхронном методе случился эксепшн, тогда он будет выброшен при вызове GetResult и мы сразу попадем в блок catch.
                    */

                    DelayDuration = int.Parse(Console.ReadLine());
                    DelayAwaiter = Task.Delay(DelayDuration).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        goto case State.WaitingForFileRead;
                    }
                    CurrentState = State.WaitingForFileRead;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    break;

                case State.WaitingForFileRead:
                    /*
                        Важно, что если выполнение идет по реально асинхронному сценарию (т. е. мы попадаем сюда не из goto case), и используется контекст синхронизации по умолчанию, либо он не задан (что по умолчанию в запросах ASP.NET Core, например), то метод MoveNext() будет вызван из какого-то потока пула потоков. То есть, разные состояния SM могут быть запущены разными потоками.
                        Обычно, нам, программистам, эта особенность не мешает. Но есть редкие кейсы, где это может быть важно - как, например, кейс в одной из задачек на самопроверку ниже в статье.
                    */
                    DelayAwaiter.GetResult();
                    ReadFileAwaiter = File.ReadAllTextAsync("file1").GetAwaiter();
                    if (ReadFileAwaiter.IsCompleted)
                    {
                        goto case State.WaitingAfterFinalDelay;
                    }
                    CurrentState = State.WaitingAfterFinalDelay;
                    Builder.AwaitUnsafeOnCompleted(ref ReadFileAwaiter, ref this);
                    break;

                case State.WaitingAfterFinalDelay:
                    // Завершение чтения файла и установка результата
                    FileContentTemp = ReadFileAwaiter.GetResult();
                    Instance._fileContent = FileContentTemp;
                    FileContentTemp = null;
                    DelayAwaiter = Task.Delay(DelayDuration).GetAwaiter();
                    if (DelayAwaiter.IsCompleted)
                    {
                        CurrentState = State.Finished;
                        return;
                    }
                    CurrentState = State.Finished;
                    Builder.AwaitUnsafeOnCompleted(ref DelayAwaiter, ref this);
                    break;
            }
        }
        catch (Exception exception)
        {
            CurrentState = State.Finished;
            Builder.SetException(exception);
        }
    }
}

private string _fileContent; // Содержимое файла

[AsyncStateMachine(typeof(AsyncStateMachine))]
public Task Main()
{
    AsyncStateMachine stateMachine = new AsyncStateMachine();
    stateMachine.Builder = AsyncTaskMethodBuilder.Create();
    stateMachine.Instance = this;
    stateMachine.CurrentState = AsyncStateMachine.State.NotStarted;
    stateMachine.Builder.Start(ref stateMachine);
    /* 
        Первый вызов MoveNext происходит прямо в stateMachine.Builder.Start. Т. е. первое состояние нашей SM фактически выполняется синхронно (и далее до первого реального асинхронного вызова).
        Исходник **
    */
    return stateMachine.Builder.Task;
}

*Method source code AsyncTaskMethodBuilder.AwaitUnsafeOnCompleted available link
** Source code for this method AsyncStateMachine.Builder.Start available link.

From the code above you can see that, in fact, for every use of the keyword await the compiler generates additional state for the state machine (SM). In addition, it is important to note that the compiler will generate the state machine itself if the modifier is used in the method type definition async.

By the way, this code won’t run because it doesn’t have some helper methods that the compiler generates. But it does help you understand how asynchrony works in C#.

Dealing with allocations

Interestingly, in Debug mode AsyncStateMachine for tasks it is presented as a class, and in Release – as a structure (struct). But even though this is a structure, if execution really follows an asynchronous scenario, under the hood in Runtime allocation will still occur for AsyncStateMachine.

When execution is asynchronous (calling DelayAwaiter.IsCompleted returns false), the CLR needs to move the state machine from the stack to the managed heap; to do this, it is boxed into AsyncStateMachineBox runtime.
For Task it happens inside AsyncTaskMethodBuilder.GetStateMachineBox.
For ValueTask this happens inside the chain (AsyncValueTaskMethodBuilder.AwaitUnsafeOnCompleted -> AsyncTaskMethodBuilder.AwaitUnsafeOnCompleted -> AsyncTaskMethodBuilder.GetStateMachineBox).

Pooling

Another interesting thing is that the CLR provides the ability pooling* AsyncStateMachineBox to minimize allocations (method StateMachineBox RentFromCache()).

* pooling in this context is a technique for reusing data to save space in the application heap and resources G.C.. If the state machine is pooled, the CLR will be able to reuse it for ValueTaskto save on heap allocations. Although, even Stephen Taub doubts in the real effectiveness of this approach.

Separately, I note that the use ValueTask often does not cancel allocations in the case of an asynchronous scenario.

Well, a final note – if you look closely at what happened to the variable delaythen we will see that it was captured and transferred to the field of the state machine (DelayDuration in clean code and <delay>5__2 in the code from the compiler). Accordingly, it is important to understand that if execution proceeds according to an asynchronous scenario (which is quite often), value type variables will be locked together with the state machine and will be moved to a managed heap – therefore Spans are prohibited from being used in methods with a modifier async.

What else to read on the topic

For example, a whole guide from Stephen Taub “How Async/Await Really Works in C#” (“How Async/Await Really Works in C#”). Translations are also available on Habré.

Or an article Prefer ValueTask to Task, always; and don’t await twicewhich describes not only the features ValueTaskbut also how to implement even asynchronous logic through IValueTaskSource or ManualResetValueTaskSourceCoreminimizing the number of memory allocations on the heap.

It’s worth mentioning that the approach from this article is similar to what Sergey Teplyakov @SergeyT did back in 2017 in his article. The difference is that in my example, SM is preserved more accurately in its original form, the code is immediately accompanied by comments inside SM, the issue with allocations is discussed, and various cases are given.

Self-test exercises

void Main()
{
    RunAsync(); //"fire-and-forget"
    Console.WriteLine("Main");
    Thread.Sleep(1500);
}

async Task RunAsync()
{
    Console.WriteLine("RunAsync 1");
    await Task.Delay(1000);
    Console.WriteLine("RunAsync 2");
}

RunAsync 1
Main
RunAsync 2

async Task Delay1()
{
    await Task.Delay(1);
}

async Task MultiDelay()
{
    var task = Task.Delay(1);
    await task;
    await task;
    await task;
}

Task Delay1()
{
    return Task.Delay(1);
}

object _syncObj = new();

async Task DelayLocked()
{
    Monitor.Enter(_syncObj);
    await Task.Delay(1);
    Monitor.Exit(_syncObj);
}

The material is relevant for version .NET 8.0.1. I will be happy to answer your questions in the comments.

In conclusion, I would like to thank Mark Shevchenko @markshevchenko and Evgeniy Peshkov @epeshk for reviewing this article. Also, in future versions of .NET async/await Maybe be greatly transformed.