contract tests with Pact in .NET. Part two

The first part of the topic covered the theory and process of contract testing of HTTP interactions. In this article we will take a closer look at testing asynchronous communications and also get acquainted with the PactBroker tool.

Using RabbitMq as an example, we will define a list of possible contract components. Depending on the specific tool, the list can vary greatly:

• Message model (structure, data types, format)

• Name, type of exchanger

• Queue name

• Settings like routing key, topics, reply-to, etc.

Materials for demo

• .NET, Pact library supports .netstandart2.0, demo uses .NET 6;

• PactNet 5.0.0-beta.2 and PactNet.Abstractions 5.0.0-beta.2 for writing tests; The reason for using the pre-release version is that the latest stable release of the library is version 4.5.0 does not support non-ASCII characters. Also, before version 5.xx, Newthonsoft.Json was used as the default serializer instead of the more modern System.Text.Json;

• EasyNetQ 7.8.0 and EasyNetQ.Serialization.SystemTextJson 7.8.0 library for working with RabbitMq;

• Docker for running RabbitMq and PactBroker containers.

  This practice is for demonstration purposes only to illustrate how PactNet works and is not intended to be a tutorial for clean, high-performance code. All demo code is available in repositories.

Asynchronous interaction between services

To test scenarios using RabbitMq, we will add a notification about the readiness of the card to the existing functionality. Thus, the contract provider (Demo.Provider) will send a model to the queue indicating the need to send a notification to the client. In turn, the consumer (Demo.Consumer) will process the message and, depending on the value of the field ShouldBeNotifiedwill print a message to the console simulating a notification to the user.

Let’s imagine that as a result of agreeing on the contract, the following agreements were recorded:

• The message model contains the fields described in the class CardOrderSatisfiedEvent and includes: product card code, user ID and sign of the need to send a notification;

• Name of the exchanger SpecialExchangeName, type direct;

• Routing-key has the value super-routing-key.

To implement this scenario, add the following dependencies to the Consumer.Host and Provider.Host assemblies:

 <PackageReference Include="EasyNetQ" Version="7.8.0" />
 <PackageReference Include="EasyNetQ.Serialization.SystemTextJson" Version="7.8.0" />

To simplify things, we implement sending a message directly in the Demo.Provider service controller:

[HttpPost("order-satisfied/{userId}")]
public async Task<ActionResult> SendCardOrderSatisfiedEvent(string userId)
{
    var advancedBus = RabbitHutch.CreateBus("host=localhost", s =>
    {
        s.EnableConsoleLogger();
        s.EnableSystemTextJson();
    }).Advanced;
    var exchange = await advancedBus
                        .ExchangeDeclareAsync("SpecialExchangeName", "direct");
    var message = new Message<CardOrderSatisfiedEvent>(
          new CardOrderSatisfiedEvent
          {
            UserId = userId,
            CardCode = Random.Shared.Next(100)
          });
    await advancedBus.PublishAsync(exchange, "super-routing-key", false, message);
    return Ok();
}

In turn, we implement the subscription to the event from the consumer side directly in the Program class, add the following code somewhere:

var advanced = RabbitHutch.CreateBus("host=localhost:5672;username=guest;password=guest", 
  s =>
      {
        s.EnableConsoleLogger();
        s.EnableSystemTextJson();
        s.Register<ITypeNameSerializer, SimpleTypeNameSerializer>();
      }).Advanced;
var exchange = advanced.ExchangeDeclare("SpecialExchangeName", "direct");
var queue = advanced.QueueDeclare("SpecialQueueName");
advanced.Bind(exchange, queue, routingKey: "super-routing-key");
advanced.Consume<CardOrderSatisfiedEvent>(queue, (message, _) =>
    Task.Factory.StartNew(() =>
    {
        var handler = app.Services.GetRequiredService<ConsumerCardService>();
        if(message.Body.ShouldBeNotified)
            handler.PushUser(message.Body);
    }));

// BAD CODE, только для демо
class SimpleTypeNameSerializer : ITypeNameSerializer
{
    public string Serialize(Type type) => type.Name;
    public Type DeSerialize(string typeName) => typeof(CardOrderSatisfiedEvent);
}  

Typically, when working with EasyNetQ, the contract provider creates a separate nuget assembly with the required message model, since by default the message property is used for serialization and deserialization messageType. The demo in question is missing nuget–assembly, mismatch problem Type.FullName two models CardOrderSatisfiedEvent in different projects it is solved using the class SimpleTypeNameSerializer, which overrides the deserialization behavior. Simply adding a reference to the assembly with the contract is pointless: we will not be able to simulate the “development” of the contract and break the pact.

All that remains is to run RabbitMq in docker and check that the services communicate using it:

docker run --rm -d -p 15671:15671/tcp -p 15672:15672/tcp -p 25672:25672/tcp 
-p 4369:4369/tcp -p 5671:5671/tcp -p 5672:5672/tcp rabbitmq:3-management

Consumer side testing

First, let's define the concepts consumer/provider And subscriber/publishersince the terminology here is a little unclear and can be confusing. As stated in the first part, in terms of Pact consumer is considered a client, a consumer of the API. This concept also means a subscriber, event recipient, or subscriber in terms of message brokers. Despite the fact that it does useful work on the data subscriber, in asynchronous systems the supplier is publisher. It follows that in our demonstration, the Demo.Provider service is the sender of the message (publisher) and event source (provider), and the Demo.Consumer service serves as the recipient of the message (subscriber) and event consumer (consumer).

Let's create it in a folder Consumer.ContractTests/RabbitMq class CardOrderSatisfiedEventTests and fill it with the following content:

public class CardOrderSatisfiedEventTests
{
    private readonly IMessagePactBuilderV4 _pactBuilder;
    private const string ComType = "RABBITMQ";

    public CardOrderSatisfiedEventTests(ITestOutputHelper testOutputHelper)
    {
        var pact = Pact.V4(consumer: "Demo.Consumer", provider: "Demo.Provider", new PactConfig
        {
            Outputters = new[] {new PactXUnitOutput(testOutputHelper)},
            DefaultJsonSettings = new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true,
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase
            }
        });
        _pactBuilder = pact.WithMessageInteractions();
    }

    [Fact(DisplayName = "Demo.Provider присылает корректный контракт и пуш отправляется, " +
                        "когда получено событие и необходимо уведомление клиента")]
    public void CardOrderSatisfiedEvent_WhenModelCorrectAndShouldBePushed_SendsPush()
    {
        // Arrange
        var message = new
        {
            UserId = Match.Type("rabbitmqUserId"),
            CardCode = Match.Integer(100),
            ShouldBeNotified = true
        };

        _pactBuilder
            .ExpectsToReceive($"{ComType}: CardOrderSatisfiedEvent with push")
            .WithMetadata("exchangeName", "SpecialExchangeName")
            .WithMetadata("routingKey", "super-routing-key")
            .WithJsonContent(message)

            // Act
            .Verify<CardOrderSatisfiedEvent>(msg =>
            {
                // Assert
                // место для вызова IConsumer.Handle и проверки логики работы обработчика
                //_consumerCardService.Verify(x => x.PushUser(msg), Times.Once);
            });
    }
    
    [Fact(DisplayName = "Demo.Provider присылает корректный контракт и пуш не отправляется, " +
                        "когда получено событие и не нужно уведомление клиента")]
    public void CardOrderSatisfiedEvent_WhenModelCorrectAndShouldNotBePushed_DontSendPush()
    {
        // Arrange
        var message = new
        {
            UserId = Match.Type(string.Empty),
            CardCode = Match.Integer(100),
            ShouldBeNotified = false
        };

        _pactBuilder
            .ExpectsToReceive($"{ComType}: CardOrderSatisfiedEvent no push")
            .WithMetadata("exchangeName", "SpecialExchangeName")
            .WithMetadata("routingKey", "super-routing-key")
            .WithJsonContent(message)

            // Act
            .Verify<CardOrderSatisfiedEvent>(msg =>
            {
                // Assert
                // место для вызова IConsumer.Handle и проверки логики работы обработчика
                //_consumerCardService.Verify(x => x.PushUser(msg), Times.Never);
            });
    }
}

• Instead of IPactBuilderV4 used IMessagePactBuilderV4, which defines pacts for systems communicating using message brokers. An object is created by calling a method WithMessageInteractions(). The rest of the configuration code is the same as what was used in the HTTP tests;

• Method ExpectsToReceive() by analogy with UponReceiving() defines the name of the test, and the familiar method WithJsonContent() defines the structure and content of the event model. At the same time, calling the method WithMetadata() allows you to capture other message artifacts such as headers, properties and other settings. In our case, the test expects the message sender to use an exchanger called SpecialExchangeName and a super-routing-key topic;

• Synchronous Verify everything is also responsible for generating the pact.json file, but, unlike the HTTP version, here you do not need to raise the server, and in the section Assert You can check the operation of the message handler.

Overall, during testing event-driven systems, Pact abstracts away the concept of message brokers and does not imply real asynchronous communication during testing. The main focus is on matching the event model and partly on its validation. Due to this generalization of brokers, Pact does not provide specific methods for working with each of them and can only offer a method WithMetadata().

Since values ​​for entities such as names of exchangers and topics are most often stored only where they are directly used, checking their compliance with the contract becomes more difficult. Duplicating their values ​​in the test (as in our example) will most likely be completely forgotten, reading from IConfiguration will require additional effort, and the dependence of the values ​​on the environment (dev, test, prod) only makes this whole situation worse. Therefore, building trust in a test that checks these artifacts is quite difficult.

Vendor-side testing

Now let's create it in the folder Provider.ContractTests/RabbitMq class ContractWithConsumerTests and fill it with the following content:

public class ContractWithConsumerTests : IDisposable
{
    private readonly PactVerifier _pactVerifier;
    private const string ComType = "RABBITMQ";

    private readonly JsonSerializerOptions _jsonSerializerOptions = new()
    {
        PropertyNameCaseInsensitive = true,
        PropertyNamingPolicy = JsonNamingPolicy.CamelCase
    };

    public ContractWithConsumerTests(ITestOutputHelper testOutputHelper) 
    {
        _pactVerifier = new PactVerifier("Demo.Provider", new PactVerifierConfig
        {
            Outputters = new []{ new PactXUnitOutput(testOutputHelper) }
        });
    }
    
    [Fact(DisplayName = "RabbitMq контракты с потребителем Demo.Consumer соблюдаются")]
    public void Verify_RabbitMqDemoConsumerContacts()
    {
        // Arrange
        var userId = "rabbitUserId";
        var cardCode = 100;
        var metadata = new Dictionary<string, string>
        {
            {"exchangeName", "SpecialExchangeName"},
            {"routingKey", "super-routing-key"}
        };
        _pactVerifier.WithMessages(scenarios =>
            {
                scenarios.Add($"{ComType}: CardOrderSatisfiedEvent with push", builder =>
                {
                    builder.WithMetadata(metadata).WithContent(() => new CardOrderSatisfiedEvent
                    {
                        UserId = userId, CardCode = cardCode, ShouldBeNotified = true
                    });
                });
                scenarios.Add($"{ComType}: CardOrderSatisfiedEvent no push", builder =>
                {
                    builder.WithMetadata(metadata).WithContent(() => new CardOrderSatisfiedEvent
                    {
                        UserId = userId, CardCode = cardCode, ShouldBeNotified = false
                    });
                });
            }, _jsonSerializerOptions)
            .WithFileSource(new FileInfo(@"..\..\..\pacts\Demo.Consumer-Demo.Provider.json"))
            
            // Act && Assert
            .WithFilter(ComType)
            .Verify();
    }

    public void Dispose()
    {
        _pactVerifier?.Dispose();
    }
}          

Instead of the one called earlier WithHttpEndpoint()which used the application we were running nearby, the method WithMessages() selects the first free port and is responsible for raising the mock host at http://localhost:port/pact-messages/. This method also accepts a set of scripts, each of which includes a title, metadata, and message body. This decision is due to the lack of a real message broker as part of testing with Pact. We simply create an abstraction in the form of a MessageProvider and fill it with our events. When running a test, this virtual broker will check the messages stored in it against the input models from the pact.json file and return the result when the method is called Verify(). Also, since the package file now contains both synchronous and asynchronous interactions, calling the method WithFilter() allows you to check only the latest ones.

Let's look at pact.json and break the API again

As a result of running the test on the Demo.Provider side, two more interactions will be added to the file already known to us, the structure of which is generally similar to the previous examples. The main differences include the contents of the section that we have already defined metadataas well as another type of interaction in the section type.

{
  "contents": {
    "content": {
      "cardCode": 100,
      "shouldBeNotified": true,
      "userId": "rabbitmqUserId"
    },
    "contentType": "application/json",
    "encoded": false
  },
  "description": "RABBITMQ: CardOrderSatisfiedEvent with push",
  "matchingRules": {
    "body": {
      "$.cardCode": {
        "combine": "AND",
        "matchers": [{"match": "integer"}]
      },
      "$.userId": {
        "combine": "AND",
        "matchers": [{"match": "type"}]
      }
   }
},
  "metadata": {
    "exchangeName": "SpecialExchangeName",
    "routingKey": "super-routing-key"
},
    "pending": false,
    "type": "Asynchronous/Messages"
},

{"description": "RABBITMQ: CardOrderSatisfiedEvent no push"...}

There are no significant differences in behavior from the HTTP test even when unapproved changes are made to the contract. So, if you make any changes to the model or metadata, Pact will throw an error like the following:

Failures:

1) Verifying a pact between Demo.Consumer and Demo.Provider - RABBITMQ: CardOrderSatisfiedEvent with push
    1.1) has a matching body
           $.userId -> Expected 'rabbitmqUserId' (String) to be equal to 'diffUserId' (String)
           $ -> Actual map is missing the following keys: cardCode

    1.2) has matching metadata
           Expected message metadata 'routingKey' to have value '"super-routing-key"' but was '"diff-super-routing-key"'
           Expected message metadata 'exchangeName' to have value '"SpecialExchangeName"' but was '"DiffSpecialExchangeName"'

Getting to know PactBroker

Based on everything abovewhat was said done and the material of the first part, by now we have two services with contract tests for each of them, covering both interactions via HTTP and those relying on RabbitMq. However, we still copy the Demo.Consumer-Demo.Provider.json file from project to project, which is not very convenient.

Fortunately, a ready-made application, PactBroker, can take on the role of pact deliverer. As the name implies, the main purpose of using this tool is the automated delivery of the pact.json file, however, it also provides a fairly informative panel for viewing existing pacts.

To operate, PactBroker requires a database to store existing contracts. In the official documentation All information about the options for launching PactBroker is presented, but we will raise an instance using docker-compose presented below. The file describes the launch of a PostgreSQL 15 DBMS cluster, as well as the broker instance that depends on it.

version: "3.9"

services:
  postgres:
    image: postgres:15
    container_name: pact-postgres
    ports:
      - "5432:5432"
    healthcheck:
      test: psql postgres -U postgres --command 'SELECT 1'
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres
      POSTGRES_DB: postgres
  
  broker:
    image: pactfoundation/pact-broker:latest-multi
    container_name: pact-broker-1
    depends_on:
      - postgres
    ports:
      - "9292:9292"
    restart: always
    environment:
      PACT_BROKER_ALLOW_PUBLIC_READ: "false"
      PACT_BROKER_BASIC_AUTH_USERNAME: admin
      PACT_BROKER_BASIC_AUTH_PASSWORD: pass
      PACT_BROKER_DATABASE_URL: "postgres://postgres:postgres@postgres/postgres"

    healthcheck:
      test: ["CMD", "curl", "--silent", "--show-error", "--fail",
             "http://pactbroker:pactbroker@localhost:9292/diagnostic/status/heartbeat"]
      interval: 1s
      timeout: 2s
      retries: 5

As a result of executing this file, the database and broker applications are launched.

We screw in automated delivery of packages

Saving generated pacts in PactBroker

Although the PactNet library provides the ability to receive pacts from the broker (which we will see very soon), the ability to send them to it lost. Subjectively, the correct solution in the environment for a real application is a separate step of sending generated pacts using pact-cli. But since a review of pact-cli is beyond the scope of this material, in our demo we use a rather controversial solution, but more understandable for demonstration purposes.

Let's create it in a folder shared new class library project. In our case, the generated files will be sent to the broker at the end of all class tests. To achieve this goal we use the interface IClassFixture and method Dispose(). PactBroker provides a list of API methods for working with it, which can be found in the broker panel. To send pacts we will use the method pacts/provider/{provider}/consumer/{consumer}/version/{consumerVersion}.

Let's create a class for sending packages and call it PactBrokerPublisher. In the demo we will limit ourselves to simple logic; the essence of the class comes down to calling the PUT method along the path presented above:

private readonly HttpClient _httpClient;
    
public PactBrokerPublisher(HttpClient httpClient) {_httpClient = httpClient;}
    
public async Task Publish(string consumer, string provider, string content, string consumerVersion)
{
    var response = await _httpClient
    .PutAsync($"pacts/provider/{provider}/consumer/{consumer}/version/{consumerVersion}",
    new StringContent(content)
    {
      Headers = { ContentType = new MediaTypeHeaderValue("application/json") }
    });

    if (response.IsSuccessStatusCode == false)
        throw new ArgumentNullException($"Ошибка во время отправки пакта в PactBroker: {response.StatusCode}");
}

To send packages at the end of running tests for the entire class, create a class PactBrokerFixture and implement the interface in it IDisposable. The purpose of the class is to send a package file to PactBroker when calling the method Dispose().

private readonly Uri _pactBrokerUri = new ("http://localhost:9292");
private readonly string _pactUsername = "admin";
private readonly string _pactPassword = "pass";
private readonly PactBrokerPublisher _pactBrokerPublisher;

public string ConsumerVersion { get; set; } 
public IPact? PactInfo { get; set; }
    
public PactBrokerFixture()
{
    var baseAuthenticationString = Convert.ToBase64String(System.Text.Encoding.ASCII.GetBytes($"{_pactUsername}:{_pactPassword}"));
    _pactBrokerPublisher = new PactBrokerPublisher(new HttpClient
    {
        DefaultRequestHeaders =
        {
            Authorization = new AuthenticationHeaderValue("Basic", baseAuthenticationString)
        },
        BaseAddress = _pactBrokerUri
    });
}
    
public void Dispose()
{
    Task.Run(async () =>
    {
        var versionSuffix = Guid.NewGuid().ToString().Substring(0, 5);
        var pactJson = await File.ReadAllTextAsync($"{PactInfo.Config.PactDir}/{PactInfo.Consumer}-{PactInfo.Provider}.json");
        await _pactBrokerPublisher.Publish(
                consumer: PactInfo.Consumer, provider: PactInfo.Provider, content: pactJson,
                $"{ConsumerVersion}-{versionSuffix}");
    });
}

All that's left to do is to complete the following steps:

1. The OrderCardTests and CardOrderSatisfiedEventTests classes implement the IClassFixture interface and also inject the PactBrokerFixture dependency into the constructor.

2. The Consumer.Domain assembly has a version tag 1.0.0.

3. The constructors of the OrderCardTests and CardOrderSatisfiedEventTests classes write values ​​to the fixture properties: ConsumerVersion and PactInfo.

brokerFixture.PactInfo = pact;
brokerFixture.ConsumerVersion = Assembly
    .GetAssembly(typeof(CardOrderSatisfiedEvent))?
    .GetCustomAttribute<AssemblyInformationalVersionAttribute>()?
    .InformationalVersion;

The main disadvantage of using this approach to sending packages is the class itself PactBrokerFixture. Since such a class itself implies the presence of only a default constructor, its initialization must be performed in the test class constructor. In addition, in our demo, to reduce the amount of code, parameters such as the broker's address and credentials are duplicated directly in the class PactBrokerFixture. However, in a real project, where these parameters will be variable, such a solution will not work, which again refers us to the individual steps during application deployment. However, the credentials can be placed in the IConfiguration of the test project, and such a solution may take root.

Receiving pacts from PactBroker

To download existing pacts, the PactNet library provides a method WithPactBrokerSource()which we will add a call to in our two tests on the provider side.

_pactVerifier
    ...
    .WithPactBrokerSource(new Uri("http://localhost:9292"), options =>
    {
        options.BasicAuthentication("admin", "pass");
        options.PublishResults(_providerVersion + $" {Guid.NewGuid().ToString().Substring(0, 5)}");
    })
    // .WithFileSource(new FileInfo(@"..\..\..\pacts\Demo.Consumer-Demo.Provider.json"))
    ...

Method BasicAuthentication() is responsible for authentication in PactBroker, the credentials for which were set at the time of raising the containers. In turn, the method PublishResult() it is not necessary to call, since it is only needed to display the results of contract verification by the supplier in the PactBroker panel. Field _providerVersion is filled in similar to the ConsumerVersion we saw earlier, but the version tag already belongs to the Provider.Contracts assembly.

PactBroker panel overview

Finally, both projects are covered with contract tests and the generated pacts are delivered using PactBroker. Let's run the tests in the folder sequentially consumer And provider. If all checks have passed, then opening the page in the browser http://localhost:9292/you can see a table displaying the pacts available to PactBroker.

The table, according to the column names, displays: the name of the contract consumer, the name of the contract provider, the date of the last publication by the consumer, the presence of a web hook (will not be discussed in this article) and the date of the last check by the supplier. The last column, depending on whether the verification has been completed on the supplier’s side, is colored: red – the verification has not been passed, yellow – the verification has not yet been carried out and green – the contract has been successfully verified on the supplier’s side.

The broker stores pacts in the database:

When you click on the document icon, a view of the pacts opens. Unfortunately, when using PactV4, the pact file for some reason is not converted to an easy-to-read format and is simply displayed as a JSON file. At the same time, previous versions, like PactV3, are successfully parsed. Compare:

Even though the second option is easier to read, the PactV4 option is still quite informative. However, if you do not have to work with Cyrillic, you can use more stable versions of the PactNet library, in which the display of pacts will look more beautiful.

Let's move on to the matrix of contracts between systems. It displays dependencies between systems, as well as verification results. As we can see, the contract between Demo.Consumer version 1.0.0-d1549 and Demo.Provider versions 1.0.0 respected by both parties. But, if the contract supplier suddenly makes some uncoordinated change to the contract, then the pact between the systems will be broken. So, Demo.Provider versions 2.0.0 and Demo.Consumer versions 1.0.0-d1549 will no longer be able to work without errors.

When you click on the value in the Pact verified column, you can see an error that was also displayed in the application console:

To complete the review of the broker panel, let’s go into the settings of any of the participants by clicking on his name from the home page. One interesting thing you can see here is a graph of dependencies between services:

Conclusion

For now, that's all I'd like to share regarding the PactNet tool. In my opinion, this library provides quite powerful tools for writing and supporting truly useful test scripts. Despite the fact that the volume of material in two articles turned out to be considerable, this is not all the capabilities of Pact. In particular, the following possibilities remained unconsidered:

• ProviderState – functionality for setting a certain state to the provider before testing. For example, checking the status of an order implies that the supplier already stores the entity of a particular order. In order not to maintain a large amount of test data on the supplier side of tests, it is possible to set the state of the second participant directly in the consumer test. Example implementations of such a scenario can be found in the PactNet library repository;

• Branches, tags – Pact has support for code branching, the best use of which is revealed in conjunction with the use of pact-cli;

• pact-cli;

• GraphQL API;

• WebHooks;

• Matchers – implementation for .NET is still somewhat crude compared to PactJS;

• The remaining PactBroker API methods, with the help of which, in theory, you can construct a flexible solution without using PactNet at all;

• There is a lot more regarding configuration, reading packages, etc.

Contract tests are not mandatory, but sometimes they do help to detect breaking changes at an early stage. Of course, like any tool, these types of tests should be used wisely.

The .NET implementation of the Pact library provides all the basic capabilities for writing contract tests out of the box. Its main disadvantages include the lack of support for in-memory TestServer, the lack of detailed documentation (only ready-made implementation examples) and the widespread use of the dynamic type, which in principle is determined by the implementation of the library. Despite all the above disadvantages, Pact still has more advantages, and I hope from the two articles it became clear what they are.