How to write tests correctly? (Part 1)

The first part is a short article without any connection to a programming language, which is suitable for anyone who has questions about how to improve the quality of their tests.

Let's discuss three basic approaches to testing something in a program:

1. Testing of observed behavior (black box principle).

Of course, as developers we know what code we write, even if we write tests using TDD. The idea here is not to use this information in the test.

Such tests are not tied to implementation details, which means that when the details change, the tests will not need to be changed. When the final result does not change (is correct), but the test fails, this is called a false positive (fragile tests that are tied to implementation details).

2. Testing the state

It also happens that the object we call mutates its state non-process dependencies, such as databases. In this case, the tested object will not return us enough data to evaluate its operation, since its observable behavior is hidden from the client.

This leads to the fact that we start using implementation details (tests become more fragile) of the database in the test. In this case, this is knowledge about the database dialect, table names, column types, etc. This data may undergo changes over time.

What to do to improve the situation? In many cases, you can give preference to other methods of the tested object (as one of the options), and then the implementation details in the test will again become unknown.

3. Behavior check

The most fragile tests are those that check:

How many times was the method called?
What parameters were passed to which place?
In what order were the calls made?
etc.

Despite the fact that such tests are the most fragile, because with almost any change in details the test will break, even if the final result is correct (false positives), they also sometimes save us: