More about the testing pyramid
The testing pyramid, also often referred to as testing levels, is grouping tests by level of detail and their purpose. This abstraction was invented by Mike Cohn and described in the book “Scrum: Agile Software Development” (Succeeding With Agile. Software Development Using Scrum).
The pyramid is broken into 4 levels (from bottom to top), for example, according to ISTQB (see wiki):
unit testing (unit)
integration testing
system testing
acceptance testing
But can be found options where 3 levels (see semaphore blog). This model combines the integration and system levels:
unit testing (unit)
integration testing (includes system testing)
acceptance testing
We can say that software development is a movement along the pyramid from the bottom up. It is important to note:
Test (manual, at high levels, or autotest, at low levels), must be at the same level as the object under test. For example, a unit test (checking functions, classes, objects, etc.) must be at the component level. It is wrong if the acceptance level runs a test that will check the minimum unit of code.
Tests at a higher level do not check logic of tests by the level/levels below.
The higher the level of the tests, the more they:
more difficult to implement, and, accordingly, more expensive to implement
more important for business and more critical for users
slow down the speed of passing test cases, for example, regression
Component level
Most often called unit testing. Less commonly referred to as unit testing. At this level, the atomic parts of the code are tested. These can be classes, functions, or class methods.
Example: Your company is developing a Calculator app that can add and subtract. Each operation is one function. Testing each function that does not depend on others is a unit test.
Unit tests find bugs at fundamental levels, they are easier to develop and maintain. An important advantage of unit tests in that they are fast and when changing the code allow quick regression (make sure the new code doesn’t break the old parts of the code).
Component level test:
Always automate
There are always more unit tests than tests from other levels
Unit tests are the fastest and require the least amount of resources
Almost always, component tests do not depend on other modules. (that’s why they are unit tests) and UI systems.
In 99% of cases, the development of unit tests is carried out by the developer, when an error is found at this level, no bug reports are created. The developer finds a bug, fixes it, launches it and checks it (abstractly speaking test-driven development) and so on again until the test is passed successfully.
At the modular level developer (or autotester) uses white box method. He knows what the minimum unit of code takes and gives, and how it works.
Integration level
Check interconnection componentswhich was tested at the modular level, with other or other componentsas well as integration of the component with the system (checking work with the OS, services and services, databases, hardware, etc.). Often referred to in English articles as service test or API test.
In the case of integration tests, it is rarely necessary to have a UI to test it. Components Software or systems interact with unit under test using interfaces. This is where testing comes into play. These are API checks, service work (checking logs on the server, records in the database) etc.
Strictly speaking, testing is also involved at the modular level. Perhaps he helps design tests or during the regression he watches the run of these tests, and if something falls down, he takes action.
Separately, I note that in integration testing, performed as functional (verification according to TOR)and non-functional checks (load on a bunch of components). At this level either gray or black box is used.
Integration testing has 3 main ways testing (imagine that each module could consist of even smaller parts):
Upwards (Bottom Up Integration): all small parts of the module are assembled into one module and tested. Next, the following small modules are assembled into one large one and tested with the previous one, etc. For example, the function of posting photos to social networks. profile consists of 2 modules: loader and publisher. The loader, in turn, consists of a compression module and sending to the server. The publisher consists of a verifier (authenticates) and photo access control. In integration testing, we will assemble the loader modules and check, then we will assemble the publisher modules, check and test the interaction between the loader and the publisher.
Top down (Top Down Integration): first, we check the operation of large modules, going down below, we add modules at a lower level. During the testing phase of the levels above, the data required from the levels below is simulated.For example, we check the work of the loader and publisher. With our hands (we create a stub function), we transfer from the uploader to the publisher a photo that was allegedly processed by the compressor.
Big Bang (“Big Bang” Integration): we collect all implemented modules of all levels, integrate them into the system and test them. If something does not work or has not been completed, then we fix it or refine it.
System level
We talked about the system level in the integration level. Just note here that:
The system level will check the interaction of the software under test with the system according to functional and non-functional requirements
It is important to test on the most approximate environment that the end user will have.
Test cases at this level are prepared:
According to requirements
Possible ways to use the software
At the system level, defects such as incorrect use of system resources, unintended combinations of user-level data, incompatibility with the environment, unintended use cases, missing or incorrect functionality, inconvenience of use, etc. are detected.
At this level use black box. The integration level allows verify requirements (Check if the software meets the prescribed requirements).
Acceptance testing
Also often called E2E tests (End-2-End) or end-to-end. Requirements are validated at this level. (checking the operation of the software as a whole, not only according to the prescribed requirements, which was checked at the system level).
Requirements check produced on a set of acceptance tests. They are developed based on requirements and possible ways to use the software.
Note that acceptance tests are carried out when (1) the product has reached the required quality level and 2) the software customer is familiar with the acceptance plan (it describes a set of scenarios and tests, the date of the conduct, etc.).
Acceptance is carried out either by internal testing (optional testers) or external testing (the customer himself and not necessarily the tester).
It is important to remember that E2E tests are more difficult to automate, take longer, cost more, are more difficult to maintain, and are difficult to regress. So there should be fewer tests like this.
In custody
I want to note that moving from level to level, an understanding of whether we are doing can come. There are questions to the requirements, there are improvements – this is normal.
I invite you to read my blog in the telegram “Tester” 🙂
