Check PHP API tests against OpenAPI definitions – Laravel example
OpenAPI Is a specification that describes RESTful APIs in JSON and YAML formats that both humans and machines can understand.
OpenAPI definitions are not specific to a specific language and can be used in a variety of ways.
The OpenAPI definition can be used when generating documentation for API mapping, in code generators when creating servers and clients in different programming languages, in testing tools, and in a number of other cases. – OpenAPI specification
In this article, we will look at how to combine OpenAPI 3.0.x definitions with link tests to verify that the API is working correctly using the package OpenAPI HttpFoundation Testing…
We will do this using the example of a fresh installation. Laravelfor which we will also generate documentation Swagger UI in the package L5 Swagger…
First, I’ll say a few words about the usefulness of this approach, but if you came here only for the code, feel free to skip to the section Laravel example…
Problem
APIs have become familiar and ubiquitous, so we rejoice in any documentation that helps us reach all endpoints. Such documentation may vary in form and content, but the descriptions in it should change whenever the API changes.
Most developers take the API documentation support as extra homework after the exam has already been passed; it is boring, tedious and usually thankless. Using annotations that store code and documentation in one place can help here; but writing them is often tedious, and even the most diligent developer can miss something without his peers noticing it.
As a result, the API and its documentation will no longer match, misleading users.
Another aspect of API support is to ensure that no endpoint stops working as expected – degradation is gradual and can go unnoticed for a long time without a proper testing strategy.
To avoid this, you can implement link testing that automatically verifies that the API is working correctly and allows you to ensure that recent changes did not lead to unpredictable consequences. However, where is the guarantee that the expected results of the build test exactly match those given in the documentation?
It would be nice to find a way to make sure they are exactly the same …
Decision
Let’s say we have API documentation and link tests, but now we need to somehow reconcile the results.
The OpenAPI specification has become a popular way of describing APIs as they evolve, but even that doesn’t stop us from having to maintain those definitions. In other words, even OpenAPI does not guarantee that everything is going well.
However, OpenAPI differs from other solutions in that it can be used as a foundation for creating growing number of instrumentsthat provide much more value from the specification than just documentation.
One such tool, written for the PHP ecosystem and maintained by the team The PHP Leagueis called OpenAPI PSR-7 Message Validator… This package for validating HTTP requests and responses against OpenAPI definitions uses the standard PSR-7…
Basically, every HTTP request and response is checked against at least one operation described in the OpenAPI definition.
Do you see where I am leading?
We just need to use this package as an additional layer on top of our assembly tests so that it checks the API responses received by the tests against the OpenAPI definitions that describe the API.
If there is no match, the test is considered to have failed.
This is how it looks in the diagram:
(Work the author)
The OpenAPI definition describes an API, and tests use it to verify that the API behaves exactly as the definition says.
As a result, the OpenAPI definition becomes the base for both code and tests, providing a single source of reliable information about the API.
PSR-7
You probably noticed a small nuance in the previous section: the OpenAPI PSR-7 Message Validator package only works with messages PSR-7, as its name suggests. The problem is that not all platforms support this standard natively. Moreover, many of them use Symfony HttpFoundation componentwhose requests and responses by default do not implement this standard.
However, the Symfony development team covered our rear by developing bridgeconverting HttpFoundation objects to PSR-7 objects if there is a suitable PSR-7 factory and PSR-17, as which they propose to use the created Tobias Nyholm implementation PSR-7…
By collecting all the needed puzzle pieces offered by the package OpenAPI HttpFoundation Testing, developers can support their link tests with OpenAPI definitions in projects that use the HttpFoundation component.
Let’s see how this is done in a Laravel project that falls under this category.
Laravel example
The code given in this section is also available as GitHub repository…
First, let’s create a new Laravel 8 project using Composer:
$ composer create-project --prefer-dist laravel/laravel openapi-example "8.*"Let’s enter the root folder of the project and install a number of subordinate packages:
$ cd openapi-example
$ composer require --dev osteel/openapi-httpfoundation-testing
$ composer require darkaonline/l5-swaggerThe first is the previously mentioned package OpenAPI HttpFoundation Testingwhich we put as a subordinate development environment package as we plan to use it as part of a test suite.
The second is the popular package L5 Swaggercreating a bridge between Laravel and platforms Swagger PHP and Swagger UI… In fact, the Swagger PHP package is not required as it uses Doctrine annotations to generate OpenAPI definitions, and we plan to write annotations by hand. But we need the Swagger UI package and it can be easily adapted to work with Laravel.
To prevent the Swagger PHP package from overwriting the OpenAPI definition, set in the file, .envin the root folder of the project, the following environment variable
L5_SWAGGER_GENERATE_ALWAYS=falseLet’s create a file named api-docs.yaml in the folder we created storage/api-docs and add the following content to it:
openapi: 3.0.3
info:
title: OpenAPI HttpFoundation Testing Laravel Example
version: 1.0.0
servers:
- url: http://localhost:8000/api
paths:
'/test':
get:
responses:
'200':
description: Ok
content:
application/json:
schema:
type: object
required:
- foo
properties:
foo:
type: string
example: barThis is a simple OpenAPI definition for a single operation – GET
Endpoint request /api/testwhich should return a JSON object with the required key foo…
Let’s check if Swagger UI renders our OpenAPI definition correctly. Let’s start the PHP development server by entering the command artisan in the root folder of the project:
$ php artisan serveOpen the path localhost: 8000 / api / documentation in the browser and replace api-docs.json on the api-docs.yaml in the top navigation bar (so Swagger UI loads the YAML definition instead of JSON, which we won’t have).
Press the key Enter or click the button Explore – our OpenAPI definition should be generated as Swagger UI documentation:
Expand the end point /test and try to open it – an error should appear 404 Not Foundsince we haven’t implemented it yet.
Let’s fix it. Open the file routes/api.php and change the example route to the following:
Route::get('/test', function (Request $request) {
return response()->json(['foo' => 'bar']);
});Go back to the Swagger UI tab and try the endpoint again – it should now return a successful response.
Let’s move on to writing a test! Open the file tests/Feature/ExampleTest.php and replace its contents with the following:
<?php
namespace TestsFeature;
use OsteelOpenApiTestingValidatorBuilder;
use TestsTestCase;
class ExampleTest extends TestCase
{
/**
* A basic test example.
*
* @return void
*/
public function testBasicTest()
{
$response = $this->get('/api/test');
$validator = ValidatorBuilder::fromYaml(storage_path('api-docs/api-docs.yaml'))->getValidator();
$result = $validator->validate($response->baseResponse, '/test', 'get');
$this->assertTrue($result);
}
}Let’s take a quick look at it. For those who don’t know Laravel $this->get() represents the testing method provided by the trait MakesHttpRequestswhich essentially makes a GET request to the specified endpoint within the application. It returns a response that is identical to the one we would receive if we made the same request from the outside.
Then we create a validator using the class OsteelOpenApiTestingResponseValidatorBuilderto which we pass the previously written YAML definition using a static method fromYaml (function storage_path returns the path to the folder storagewhere the definition is stored).
If we were working with a JSON definition, we would use the method fromJson; in addition, both methods accept both strings and YAML and JSON files.
The builder returns an instance OsteelOpenApiTestingResponseValidator, in which we call the GET method, passing the path and response as parameters ($response represents a wrapper object IlluminateTestingTestResponsefor indoor object HttpFoundationwhich can be accessed via a public property baseResponse).
This code seems to say: “I want to make sure this answer matches the OpenAPI definition for a GET request for the / test path“.
It can be written like this:
$result = $validator->get('/test', $response->baseResponse);This is acceptable because the validator supports abbreviations for every HTTP method supported by OpenAPI (GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS, and TRACE) to make it easier to test responses for the corresponding operations.
Please note that the specified path must exactly match one of the ways OpenAPI definitions.
Now you can run the test, which will complete successfully:
$ ./vendor/bin/phpunit tests/FeatureOpen the file again routes/api.php and change your route to the following:
Route::get('/test', function (Request $request) {
return response()->json(['baz' => 'bar']);
});Run the test again. It should now fail as the response contains baz instead fooand the latter is expected by OpenAPI’s definition.
Our test is officially supported by OpenAPI!
Of course, this is a very simplified example, intended only for demonstration, but in a real situation it will be correct to rewrite MakesHttpRequests method call trait so that it performs both the test request and validation according to OpenAPI.
As a result, our test will fit in one line:
$this->get('/api/test');It can be implemented as a new trait MakesOpenApiRequestswhich will “expand” the trait MakesHttpRequests, as a result of which it will first call the parent method call for an answer. It will extract the path from the URI and check the response against the OpenAPI definition before returning it, and then invoke the test to perform additional checks.
Conclusion
The above approach significantly improves the reliability of the API, but it is not a universal solution – the assembly tests must cover each individual endpoint, which is not easy to automate, and the developers must still remain disciplined and attentive. At first, this may even be perceived as coercion, since the developers are essentially make maintain documentation in order to write successful tests.
But as a result, the documentation is guaranteed to be more accurate, and users will enjoy fewer errors related to the API, which in turn will reduce the frustration of developers who have to spend less time looking for bloopers.
By making OpenAPI definitions the single benchmark for both API documentation and build tests, we motivate developers to keep them up to date, which will naturally become a priority.
As for maintaining the OpenAPI definitions themselves, doing it manually is more difficult. Annotations can be used, but I prefer to support the YAML file directly. Various IDE extensions, for example this is for VSCode, greatly simplify this task, but if the very appearance of the YAML or JSON file disgusts you, you can work with it using the pleasant interface of special instruments, such as Stoplight Studio…
Since we mentioned Stoplight *, I can recommend the article Where to start, API design or code (API Design-First vs Code First) written by By Phil Sturgeonas a great starting point for API documentation to help you choose the approach you like.
* I have nothing to do with Stoplight.
Resources
Translation of the article prepared as part of the course “Framework Laravel”… Original author Yannick chenot… If you are interested in learning about the training format and the course program in more detail, we invite you to Open Day online.
