Writing API autotests in TypeScript + Playwright

Introduction

In this article, we will analyze the Autotests API in TypeScript. As a framework, we will choose playwright.

We want our autotests to meet the following requirements:

  1. The checks must be complete, that is, we must check the status of the response code, the data in the response body, validate the JSON schema;

  2. Test data preparation should be at the fixture level;

  3. Clear and beautiful report;

Requirements

To write the autotest API, we will use:

  • playwright – yarn add playwright/npm install playwright;

  • allure-playwright – yarn add allure-playwright/npm install allure-playwright;

  • dotenv – yarn add dotenv /npm install dotenv, – to read settings from .env file;

  • ajv- yarn add ajv/npm install ajv, – to validate the JSON schema;

We will write tests for the public API https://api.sampleapis.com/futurama/questions. This API is just an example. On real projects, the API can be much more complicated, but the essence of writing autotests remains the same.

Settings

Add basic project settings to the .env file

CI=1 # For playwright

ENV_NAME="Local" # Name of our env just for example, can be "Dev", "Staging" etc.

ALLURE_RESULTS_FOLDER="allure-results" # Folder where allure results are stored

BASE_URL="https://api.sampleapis.com" # API endpoint
TEST_USER_EMAIL="some@gmail.com" # Some random user just for example
TEST_USER_PASSWORD="some" # Some random password just for example

The playwright config file will look like a standard one, just add allure-report. Let’s exclude settings that are unnecessary for API tests, such as projects, headless, video, screenshot. You can read more about the playwright configuration here.

playwright.config.ts

import { defineConfig } from '@playwright/test';
import { config as dotenvConfig } from 'dotenv';
import { resolve } from 'path';

dotenvConfig({ path: resolve(__dirname, '.env'), override: true });

/**
 * Read environment variables from file.
 * https://github.com/motdotla/dotenv
 */
// require('dotenv').config();

/**
 * See https://playwright.dev/docs/test-configuration.
 */
export default defineConfig({
  testDir: './tests',
  /* Maximum time one test can run for. */
  timeout: 30 * 1000,
  expect: {
    /**
     * Maximum time expect() should wait for the condition to be met.
     * For example in `await expect(locator).toHaveText();`
     */
    timeout: 5000
  },
  /* Run tests in files in parallel */
  fullyParallel: true,
  /* Fail the build on CI if you accidentally left test.only in the source code. */
  forbidOnly: !!process.env.CI,
  /* Retry on CI only */
  retries: process.env.CI ? 2 : 0,
  /* Opt out of parallel tests on CI. */
  workers: process.env.CI ? 1 : undefined,
  /* Reporter to use. See https://playwright.dev/docs/test-reporters */
  reporter: [['html'], ['allure-playwright']],
  globalTeardown: require.resolve('./utils/config/global-teardown'),
  /* Shared settings for all the projects below. See https://playwright.dev/docs/api/class-testoptions. */
  use: {
    /* Maximum time each action such as `click()` can take. Defaults to 0 (no limit). */
    actionTimeout: 0,
    /* Base URL to use in actions like `await page.goto('/')`. */
    // baseURL: 'http://localhost:3000',

    /* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
    trace: 'on-first-retry'
  }
});

Let’s pay attention to the line:

dotenvConfig({ path: resolve(__dirname, '.env'), override: true });

Here we load the settings and the .env file we created earlier.

Life hack. If you have multiple environments (dev, test, staging, local), then you can create multiple .env files, for example, .env.dev, .env.test, .env.staging, put settings for a specific environment in each of them. Then loading the file with the settings will look something like this:

dotenvConfig({ path: resolve(__dirname, process.env.ENV_FILE), override: true });

variable ENV_FILE will have to be added to the environment or to the launch command in advance export ENV_FILE=".env.test" && npx playwright test

types

Let’s write types for the question object from the API https://api.sampleapis.com/futurama/questions. The object itself looks like this:

{
  "id": 1,
  "question": "What is Fry's first name?",
  "possibleAnswers": [
    "Fred",
    "Philip",
    "Will",
    "John"
  ],
  "correctAnswer": "Philip"
}

utils\types\api\questions.ts

export interface Question {
  id: number;
  question: string;
  possibleAnswers: string[];
  correctAnswer: string | number;
}

export interface UpdateQuestion extends Partial<Omit<Question, 'id'>> {}

utils\types\api\authentication.ts

export interface AuthUser {
  email: string;
  password: string;
}

export interface APIAuth {
  authToken?: string;
  user?: AuthUser;
}

AuthUser let’s just take it as an example (your project may have other requirements for authentication).

utils\types\api\client.ts

import { APIRequestContext } from '@playwright/test';

export interface APIClient {
  context: APIRequestContext;
}

APIClient we will need to implement the client API. We will talk about this below when we describe clients.

Context

playwright has a notion context, which can be used to make API requests. Using the context, we can set the baseURL, headers, for example, an authorization token, proxy, timeout, read more here.

First, let’s write a basic context that will be used for all requests without authorization

core\context\default-context.ts

import { request } from '@playwright/test';

export const getDefaultAPIContext = async () => {
  return await request.newContext({
    baseURL: process.env.BASE_URL
  });
};

Now let’s write the context that will be used to make requests to the authenticated API. In this API https://api.sampleapis.com/futurama/questions there is no authentication, I specified the API Key authentication header for the sake of the example. Most likely on your project you will have a different header for authentication.

core\context\auth-context.ts

import { APIRequestContext, request } from '@playwright/test';
import { APIAuth } from '../../utils/types/api/authentication';
import { getAuthAPIClient } from '../api/authentication-api';

export const getAuthAPIContext = async ({ user, authToken }: APIAuth): Promise<APIRequestContext> => {
  let extraHTTPHeaders: { [key: string]: string } = {
    accept: '*/*',
    'Content-Type': 'application/json'
  };
API endpoints
  if (!user && !authToken) {
    throw Error('Provide "user" or "authToken"');
  }

  if (user && !authToken) {
    const authClient = await getAuthAPIClient();
    const token = await authClient.getAuthToken(user);

    extraHTTPHeaders = { ...extraHTTPHeaders, Authorization: `Token ${token}` };
  }
  if (authToken && !user) {
    extraHTTPHeaders = { ...extraHTTPHeaders, Authorization: `Token ${authToken}` };
  }

  return await request.newContext({
    baseURL: process.env.BASE_URL,
    extraHTTPHeaders
  });
};

API Clients

Now let’s describe the clients for interacting with the API.

For example, let’s describe methods that will work with authentication. For https://api.sampleapis.com/futurama/questions authentication is not required, but in your project you can specify your methods to get the token.

core\api\authentication-api.ts

import test, { APIRequestContext, APIResponse } from '@playwright/test';
import { APIRoutes } from '../../utils/constants/routes';
import { APIClient } from '../../utils/types/api/client';
import { AuthUser } from '../../utils/types/api/authentication';
import { getDefaultAPIContext } from '../context/default-context';

class AuthAPIClient implements APIClient {
  constructor(public context: APIRequestContext) {}

  async getAuthTokenApi(data: AuthUser): Promise<APIResponse> {
    const stepName = `Getting token for user with email "${data.email}" and password "${data.password}"`;

    return await test.step(stepName, async () => {
      return await this.context.post(APIRoutes.Auth, { data });
    });
  }

  async getAuthToken(data: AuthUser): Promise<string> {
    // Should be used like this:

    // const response = await this.getAuthTokenApi(data);
    // const json = await response.json();

    // expect(response.status()).toBe(200);

    // return json.token;

    return 'token';
  }
}

export const getAuthAPIClient = async (): Promise<AuthAPIClient> => {
  const defaultContext = await getDefaultAPIContext();
  return new AuthAPIClient(defaultContext);
};

Please note that we are implementing APIClient and prescribe context in the constructor. Next, we will pass the context we need inside the client and with the help of the client we will execute requests.

Client for questions:

import test, { APIRequestContext, APIResponse } from '@playwright/test';
import { expectStatusCode } from '../../utils/assertions/solutions';
import { APIRoutes } from '../../utils/constants/routes';
import { APIClient } from '../../utils/types/api/client';
import { Question, UpdateQuestion } from '../../utils/types/api/questions';

export class QuestionsAPIClient implements APIClient {
  constructor(public context: APIRequestContext) {}

  async getQuestionAPI(questionId: number): Promise<APIResponse> {
    return await test.step(`Getting question with id "${questionId}"`, async () => {
      return await this.context.get(`${APIRoutes.Questions}/${questionId}`);
    });
  }

  async getQuestionsAPI(): Promise<APIResponse> {
    return await test.step('Getting questions', async () => {
      return await this.context.get(APIRoutes.Questions);
    });
  }

  async createQuestionAPI(data: Question): Promise<APIResponse> {
    return await test.step(`Creating question with id "${data.id}"`, async () => {
      return await this.context.post(APIRoutes.Questions, { data });
    });
  }

  async updateQuestionAPI(questionId: number, data: UpdateQuestion): Promise<APIResponse> {
    return await test.step(`Updating question with id "${questionId}"`, async () => {
      return await this.context.patch(`${APIRoutes.Questions}/${questionId}`, { data });
    });
  }

  async deleteQuestionAPI(questionId: number): Promise<APIResponse> {
    return await test.step(`Deleting question with id "${questionId}"`, async () => {
      return await this.context.delete(`${APIRoutes.Questions}/${questionId}`);
    });
  }

  async createQuestion(data: Question): Promise<Question> {
    const response = await this.createQuestionAPI(data);
    await expectStatusCode({ actual: response.status(), expected: 201, api: response.url() });

    return await response.json();
  }
}

Using QuestionsAPIClientwe will be able to perform simple CRUD requests to the API https://api.sampleapis.com/futurama/questions.

utilities

Let’s add the necessary utilities that will help make the tests better.

We will store the routings enum so as not to duplicate the code and clearly see which routings are used:

utils\constants\routes.ts

export enum APIRoutes {
  Auth="/auth",
  Info = '/futurama/info',
  Cast="/futurama/cast",
  Episodes="/futurama/episodes",
  Questions="/futurama/questions",
  Inventory = '/futurama/inventory',
  Characters="/futurama/characters"
}

Let’s add utilities for random data generation:

utils\fakers.ts

const NUMBERS = '0123456789';
const LETTERS = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz';
const LETTERS_WITH_NUMBERS = LETTERS + NUMBERS;

export const randomNumber = (start: number = 500, end: number = 2000): number =>
  Math.floor(Math.random() * (end - start + 1) + end);

export const randomString = (start: number = 10, end: number = 20, charSet: string = LETTERS_WITH_NUMBERS): string => {
  let randomString = '';
  for (let index = 0; index < randomNumber(start, end); index++) {
    const randomPoz = Math.floor(Math.random() * charSet.length);
    randomString += charSet.substring(randomPoz, randomPoz + 1);
  }
  return randomString;
};

export const randomListOfStrings = (start: number = 10, end: number = 20): string[] => {
  const range = randomNumber(start, end);

  return Array.from(Array(range).keys()).map((_) => randomString());
};

I used native tools to generate random string, number; this will be more than enough for us. In your own projects, you can use fakers to generate data, for example, faker-js.

Now let’s write utilities that will help us generate data to send to the API:

utils\api\questions.ts

import { randomListOfStrings, randomNumber, randomString } from '../fakers';
import { Question, UpdateQuestion } from '../types/api/questions';

export const getRandomUpdateQuestion = (): UpdateQuestion => ({
  question: randomString(),
  correctAnswer: randomString(),
  possibleAnswers: randomListOfStrings()
});

export const getRandomQuestion = (): Question => ({
  id: randomNumber(),
  question: randomString(),
  correctAnswer: randomString(),
  possibleAnswers: randomListOfStrings()
});

utils\fixtures.ts

import { Fixtures } from '@playwright/test';

export const combineFixtures = (...args: Fixtures[]): Fixtures =>
  args.reduce((acc, fixture) => ({ ...acc, ...fixture }), {});

In this example combineFixtures is a helper method that will help us combine several fixture objects into one. You can do without it, but I’m more comfortable that way.

assertions

Before we start writing tests, we need to prepare checks.

Let’s describe the basic checks that will be used throughout the project:

utils\assertions\solutions.ts

import { expect, test } from '@playwright/test';

type ExpectToEqual<T> = {
  actual: T;
  expected: T;
  description: string;
};

type ExpectStatusCode = { api: string } & Omit<ExpectToEqual<number>, 'description'>;

export const expectToEqual = async <T>({ actual, expected, description }: ExpectToEqual<T>) => {
  await test.step(`Checking that "${description}" is equal to "${expected}"`, async () => {
    expect(actual).toEqual(expected);
  });
};

export const expectStatusCode = async ({ actual, expected, api }: ExpectStatusCode): Promise<void> => {
  await test.step(`Checking that response status code for API "${api}" equal to ${expected}`, async () => {
    await expectToEqual({ actual, expected, description: 'Response Status code' });
  });
};

In fact, you can not write these wrappers, but then the report will display something like expect.toEqualwhich is not informative. Therefore, it is better to use the solution above.

Let’s add checks for questions:

utils\assertions\api\questions.ts

import { Question, UpdateQuestion } from '../../types/api/questions';
import { expectToEqual } from '../solutions';

type AssertQuestionProps = {
  expectedQuestion: Question;
  actualQuestion: Question;
};

type AssertUpdateQuestionProps = {
  expectedQuestion: UpdateQuestion;
  actualQuestion: UpdateQuestion;
};

export const assertUpdateQuestion = async ({ expectedQuestion, actualQuestion }: AssertUpdateQuestionProps) => {
  await expectToEqual({
    actual: expectedQuestion.question,
    expected: actualQuestion.question,
    description: 'Question "question"'
  });
  await expectToEqual({
    actual: expectedQuestion.correctAnswer,
    expected: actualQuestion.correctAnswer,
    description: 'Question "correctAnswer"'
  });
  await expectToEqual({
    actual: expectedQuestion.possibleAnswers,
    expected: actualQuestion.possibleAnswers,
    description: 'Question "possibleAnswers"'
  });
};

export const assertQuestion = async ({ expectedQuestion, actualQuestion }: AssertQuestionProps) => {
  await expectToEqual({ actual: expectedQuestion.id, expected: actualQuestion.id, description: 'Question "id"' });
  await assertUpdateQuestion({ expectedQuestion, actualQuestion });
};

We have written functions assertUpdateQuestion, assertQuestionso that you can use and reuse them in tests later. If we refuse this layer, then we will get a bunch of duplicates in the tests.

schema

We need to describe the JSON schema validation module. For validation, we will use the library https://ajv.js.org/guide/typescript.html

Let’s write a validator:

utils\schema\validator.ts

import test from '@playwright/test';
import Ajv, { JSONSchemaType } from 'ajv';

const ajv = new Ajv();

type ValidateSchemaProps<T> = {
  schema: JSONSchemaType<T>;
  json: T | T[];
};

export const validateSchema = async <T>({ schema, json }: ValidateSchemaProps<T>) => {
  await test.step('Validating json schema', async () => {
    const validate = ajv.compile(schema);

    if (!validate(json)) {
      const prettyJson = JSON.stringify(json, null, 2);
      const prettyError = JSON.stringify(validate.errors, null, 2);
      throw Error(`Schema validation error: ${prettyError}\nJSON: ${prettyJson}`);
    }
  });
};

Function validateSchema will accept a schema and a JSON object that must be validated.

Next, you need to describe the schema for questions:

utils\schema\api\questions-schema.ts

import { JSONSchemaType } from 'ajv';
import { Question, UpdateQuestion } from '../../types/api/questions';

export const questionSchema: JSONSchemaType<Question> = {
  title: 'Question',
  type: 'object',
  properties: {
    id: { type: 'integer' },
    question: { type: 'string' },
    possibleAnswers: { type: 'array', items: { type: 'string' } },
    correctAnswer: { anyOf: [{ type: 'string' }, { type: 'integer' }] }
  },
  required: ['id', 'question', 'correctAnswer', 'possibleAnswers']
};

export const updateQuestionSchema: JSONSchemaType<UpdateQuestion> = {
  title: 'UpdateQuestion',
  type: 'object',
  properties: {
    question: { type: 'string', nullable: true },
    possibleAnswers: { type: 'array', items: { type: 'string' }, nullable: true },
    correctAnswer: { type: 'string', nullable: true }
  }
};

export const questionsListSchema: JSONSchemaType<Question[]> = {
  title: 'QuestionsList',
  type: 'array',
  items: {
    $ref: '#/definitions/question',
    type: 'object',
    required: ['id', 'question', 'correctAnswer', 'possibleAnswers']
  },
  definitions: {
    question: {
      title: 'Question',
      type: 'object',
      properties: {
        id: { type: 'integer' },
        question: { type: 'string' },
        possibleAnswers: { type: 'array', items: { type: 'string' } },
        correctAnswer: { anyOf: [{ type: 'string' }, { type: 'integer' }] }
      },
      required: ['id', 'question', 'correctAnswer', 'possibleAnswers']
    }
  }
};

How to write a JSON schema can be found here https://json-schema.org/understanding-json-schema/. And you can generate a JSON schema, for example, here https://www.liquid-technologies.com/online-json-to-schema-converter.

Fixtures

And the last thing we need to do before writing tests is describe the fixtures.

First, let’s write a fixture to get a test user:

fixtures\users.ts

import { Fixtures } from '@playwright/test';
import { AuthUser } from '../utils/types/api/authentication';

export type UsersFixture = {
  testUser: AuthUser;
};

export const usersFixture: Fixtures<UsersFixture> = {
  testUser: async ({}, use) => {
    const email = process.env.TEST_USER_EMAIL;
    const password = process.env.TEST_USER_PASSWORD;

    if (!email || !password) {
      throw Error(`Provide "TEST_USER_EMAIL" and "TEST_USER_PASSWORD" inside .env`);
    }

    await use({ email, password });
  }
};

Now let’s write fixtures for questions:

fixtures\questions.ts

import { Fixtures } from '@playwright/test';
import { QuestionsAPIClient } from '../core/api/questions-api';
import { getAuthAPIContext } from '../core/context/auth-context';
import { getRandomQuestion } from '../utils/api/questions';
import { Question } from '../utils/types/api/questions';
import { UsersFixture } from './users';

export type QuestionsFixture = {
  questionsClient: QuestionsAPIClient;
  question: Question;
};

export const questionsFixture: Fixtures<QuestionsFixture, UsersFixture> = {
  questionsClient: async ({ testUser }, use) => {
    const authContext = await getAuthAPIContext({ user: testUser });
    const questionsClient = new QuestionsAPIClient(authContext);

    await use(questionsClient);
  },
  question: async ({ questionsClient }, use) => {
    const randomQuestion = getRandomQuestion();
    const question = await questionsClient.createQuestion(randomQuestion);

    await use(question);

    await questionsClient.deleteQuestionAPI(question.id);
  }
};

Fixture questionsClient will construct and pass the client to our tests to interact with the questions API. Fixture question will create a question object via the API and, at the end of the test, will delete the created object.

testing

Now you can write tests using all the clients, functions, fixtures, checks that were written above.

Let’s extend the standard test object from playwright and add our fixtures to it:

tests\questions-test.ts

import { test as base } from '@playwright/test';
import { questionsFixture, QuestionsFixture } from '../fixtures/questions';
import { usersFixture, UsersFixture } from '../fixtures/users';
import { combineFixtures } from '../utils/fixtures';

export const questionsTest = base.extend<UsersFixture, QuestionsFixture>(
  combineFixtures(usersFixture, questionsFixture)
);

tests\questions.spec.ts

import { getRandomQuestion, getRandomUpdateQuestion } from '../utils/api/questions';
import { assertQuestion, assertUpdateQuestion } from '../utils/assertions/api/questions';
import { expectStatusCode } from '../utils/assertions/solutions';
import { questionSchema, questionsListSchema, updateQuestionSchema } from '../utils/schema/api/questions-schema';
import { validateSchema } from '../utils/schema/validator';
import { Question } from '../utils/types/api/questions';
import { questionsTest as test } from './questions-test';

test.describe('Questions', () => {
  test('Get question', async ({ question, questionsClient }) => {
    const response = await questionsClient.getQuestionAPI(question.id);
    const json: Question = await response.json();

    await expectStatusCode({ actual: response.status(), expected: 200, api: response.url() });
    await assertQuestion({ expectedQuestion: question, actualQuestion: json });

    await validateSchema({ schema: questionSchema, json });
  });

  test('Get questions', async ({ questionsClient }) => {
    const response = await questionsClient.getQuestionsAPI();
    const json: Question[] = await response.json();

    await expectStatusCode({ actual: response.status(), expected: 200, api: response.url() });

    await validateSchema({ schema: questionsListSchema, json });
  });

  test('Create question', async ({ questionsClient }) => {
    const payload = getRandomQuestion();

    const response = await questionsClient.createQuestionAPI(payload);
    const json: Question = await response.json();

    await expectStatusCode({ actual: response.status(), expected: 201, api: response.url() });
    await assertQuestion({ expectedQuestion: payload, actualQuestion: json });

    await validateSchema({ schema: questionSchema, json });
  });

  test('Update question', async ({ question, questionsClient }) => {
    const payload = getRandomUpdateQuestion();

    const response = await questionsClient.updateQuestionAPI(question.id, payload);
    const json: Question = await response.json();

    await expectStatusCode({ actual: response.status(), expected: 200, api: response.url() });
    await assertUpdateQuestion({ expectedQuestion: payload, actualQuestion: json });

    await validateSchema({ schema: updateQuestionSchema, json });
  });

  test('Delete question', async ({ question, questionsClient }) => {
    const deleteQuestionResponse = await questionsClient.deleteQuestionAPI(question.id);
    const getQuestionResponse = await questionsClient.getQuestionAPI(question.id);

    await expectStatusCode({
      actual: getQuestionResponse.status(),
      expected: 404,
      api: getQuestionResponse.url()
    });
    await expectStatusCode({
      actual: deleteQuestionResponse.status(),
      expected: 200,
      api: deleteQuestionResponse.url()
    });
  });
});

Here are five tests for standard CRUD operations for questions API https://api.sampleapis.com/futurama/questions.

Returning to our requirements:

  1. We check the status of the response code, response body, JSON schema;

  2. Data is prepared inside fixtures;

  3. Let’s take a look at the report below.

report

Before generating the report, I want to show one interesting feature of playwright, with which we can do global setup, teardown.

IN playwright.config.ts we added this entry:

globalTeardown: require.resolve('./utils/config/global-teardown')

Which points to the path to the file from which the function is exported globalTeardown. Playwright will launch this feature at the end of the test session. For example, let’s display all environment variables in the allure report using global-teardown :

utils\reporters\allure.ts

import fs from 'fs';
import path from 'path';

export const createAllureEnvironmentFile = (): void => {
  const reportFolder = path.resolve(process.cwd(), process.env.ALLURE_RESULTS_FOLDER);
  const environmentContent = Object.entries(process.env).reduce(
    (previousValue, [variableName, value]) => `${previousValue}\n${variableName}=${value}`,
    ''
  );

  fs.mkdirSync(reportFolder, { recursive: true });
  fs.writeFileSync(`${reportFolder}/environment.properties`, environmentContent, 'utf-8');
};

utils\config\global-teardown.ts

import { FullConfig } from '@playwright/test';
import { createAllureEnvironmentFile } from '../reporters/allure';

async function globalTeardown(_: FullConfig): Promise<void> {
  createAllureEnvironmentFile();
}

export default globalTeardown;

By analogy, you can do, for example, getting a token at the beginning of a test session globalSetupand then clearing the database after the end of the test session globalTeardown.

In the report we will see the environment variables that we registered in globalTeardown

Let’s run the tests and look at the report:

npx playwright test

Now let’s run the report:

allure serve

Or you can build a report and open the index.html file in the allure-reports folder:

allure generate

See the full report here.

Conclusion

All project source code located on my github.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *