Epistemology of Component Interaction Testing

How do you know that the components are interacting correctly?

Most software systems consist of a graph of components. Let me explain: I use the word “component” in a broad sense, meaning a set of functional capabilities — it could be an object, a module, a function, a data type, or something else. Some components deal with the bigger picture, and typically coordinate the work of other components that perform more specific tasks. If you imagine a component graph as a tree, some components would be leaves.

Leaf components, being self-contained and having no dependencies, are usually the easiest to test. Most test-driven development (TDD) tasks focus on these components: tennis, bowling, rhombus, Roman numerals, “gossip-gathering drivers” and so on. Even the task c legacy security manager simple and quite self-sufficient. There is nothing wrong with that, and there are good reasons for keeping such exercises simple. After all, you want to complete the task in a few hours. This is unlikely to be possible if the task involves developing an entire website with a user interface, persistent data storage, security, data validation, well-thought-out business logic, third-party integration, email newsletters, logging, and so on.

This means that even if you master TDD for leaf functionality, you may struggle when working with higher-level components. How do you write unit tests for code that has dependencies?

Interaction Based Testing

A common solution is the principle dependency inversion. For example, you can use dependency injection (Dependency Injection) to inject test doubles into the system under test (SUT). This allows you to monitor the behavior of dependencies and verify that the system under test behaves as expected. Additionally, you can verify that the SUT interacts with dependencies as expected. This is called interaction based testing. This is perhaps the most common form of unit testing in the industry, and is well described in book “Test-Driven Object-Oriented Software Development.”

The most useful types of test doubles for interaction-based testing are stubs and mocks. However, they are problematic because they violate encapsulation. And encapsulation is known to be an important aspect in functional programming.

I have already described, how to go From Interaction-Based Testing to State-Based Testing, and Why Functional Programming Is Inherently more testable.

How to test composition of pure functions?

When you move to functional programming, you will sooner or later need to compose or orchestrate pure functions. How to check that the composition of pure functions works correctly? This can be verified using mocks or spies.

You've developed a component A, perhaps as a higher-order function, that depends on another component B. You want to test that A interacts correctly with B, but if interaction-based testing is no longer “allowed” (because it breaks encapsulation), then what do you do?

I've been pondering this question myself for a long time, while enjoying how functional programming makes most tasks easier. It took me a while to realize that the answer, as is often the case, is Mu. I'll come back to this later.

“I have a component A, which, frankly, acts as a controller, doing checks and processing around a fairly complex state. This process can have multiple outcomes, let’s call them Success, Fail, and Missing (the actual states are unimportant, but I’d like to have more than two). We also have a component B, which is responsible for rendering the result. Of course, three different states lead to three different views, but the views also depend on the state (let’s say we have a browser, mobile, and native client, and we want different views). The components are initially objects, with B having three separate methods, but I can express them as pure functions, at least for the purposes of this discussion – A, and then BSuccess, BFail, and BMissing. I can easily test each part of B separately; the problem comes when I need to test A calling different parts of B. If I’m mocking, the solution is simple – I inject a mock of B into A, and then verify that A calls the appropriate parts based on the result of the process. “This requires knowledge of the internals of A, but otherwise it's a well-known and understood approach. But if I want to avoid using mocks, what do I do? I can't test A without relying on some code path in B, and that to me means I'm losing the benefits of unit testing and moving into the realm of integration testing.”

In his letter, Sergei Rogovtsev explicitly gave me permission to quote him and raise this question. As I said, I have worked on this question myself, so I consider it worthy of attention. However, I cannot work with it without questioning its premises. This is not a criticism of Sergei Rogovtsev; after all, I have asked myself this question, so any criticism I make is directed at me as much as at myself.

Axiomatic vs. scientific knowledge

It might be useful to raise a discussion. How do we know that the software (or a subsystem of it) works? One answer to this question might be that the tests pass. If all the tests pass, we can say with a high degree of confidence that the system works.

In Sergey Rogovtsev's language, we can easily test component B because it consists of pure functions.

So how do we test component A? With mocks and stubs, we can prove that the interaction works as intended. The key word here is prove. If you assume that component B works correctly, “all” you need to do is demonstrate that component A interacts correctly with component B. I used to do this a lot and called it data flow verification or structural verification. The idea was that if you can demonstrate that component A interacts correctly with any LSP-compatible implementation of component B, and then also demonstrate that in reality (when assembled into root of composition) component A is linked with component B, which has also been demonstrated to work correctly, then the (sub)system works correctly.

It's almost like a mathematical proof. First prove it. Lemma Bthen prove Theorem Ausing Lemma B. Finally, formulate consequence C: b is a special case considered Lemma Bhence, a covered Theorem A. Which is what was required to be proven.

It is a logical and deductive approach to the problem of testing the composition of a whole from testable parts. It is almost mathematical in the sense that it attempts to construct axiomatic system.

This is also a fundamental flaw.

I didn't realize this ten years ago, and in practice it worked pretty well – except for the problems that come from poor encapsulation. The problem with this approach is that an axiomatic system is only as strong as its axioms. What are the axioms in this system? The axioms, or premises, are that each of the components (A and B) is already correct. Based on these premises, this approach to testing proves that the composition is also correct.

How do we know that the components are working correctly?

In this context, the answer is that they pass all the tests. However, this is not proofRather, it is an experimental knowledge, more like science than mathematics.

Why then are we trying? provethat the composition works correctly? Why not just check This?

This remark gets to the heart of the epistemology of testing. How do we know that software works? Typically, not proving its correctness, but by subjecting it to experiments. As I already told in the book Code That Fits in Your Headautomated tests can be viewed as scientific experiments that we repeat over and over again.

Integration testing

Let's briefly describe the arguments we've made so far: Although to check the correctness of a component's interaction with another component, you you can using mocks and spies, this may be overkill. Essentially, you are trying to prove a proposition based on dubious evidence.

Is it really important that the two components interacted right? Aren't components implementation details? Do users care about that?

Users and other interested parties are concerned behavior software system. Why not test it?

Unfortunately, this is easier said than done. Sergey Rogovtsev makes it clear that he is not a fan of integration testing. Although he does not explain why directly, there are good reasons to be wary of integration testing. As eloquently stated explained J.B. Rainsbergerthe main problem with integration testing is the combinatorial explosion of test cases. If you have to write 53,000 test cases to cover all combinations of paths through integrated components, which test cases will you write? Certainly not all 53,000.

J.B. Rainsberger's argument is that if you're going to write no more than a dozen unit tests, you're unlikely to cover enough of them to be confident that the system works.

What if, however, you can write hundreds or thousands of test cases?

Property Based Testing

You may remember that this article is based on the concept of functional programming, where property based testing is a common testing technique. While you can use this technique to some extent in object-oriented programming, it is often difficult due to side effects and non-deterministic behavior.

When you write a property-based test, you write one piece of code that evaluates property of the system under test. The property is similar to a parameterized unit test; the difference is that the input data is randomly generated, but you can control it. This allows you to write hundreds or thousands of test cases without having to write them explicitly.

So, epistemologically, you can use property-based testing with integrated components to ensure that a (sub)system works. In practice, I've found that this technique gives me the same sense of confidence as unit testing with stubs and spies.

Examples

I realize that all of this sounds abstract and theoretical. An example would be just right. However, such examples are quite complex and therefore deserve a separate article:

Sergey Rogovtsev kindly provided a rather abstract, but minimal and self-sufficient exampleI'll first break it down and then look at more realistic examples.

Conclusion

How do you know if a software system is working correctly? Ultimately, if it behaves as expected, then it is working correctly. However, testing the entire system from the outside is rarely practical. The number of possible test cases is simply too large.

A partial solution to this problem is to decompose the system into components. Then you can test the components individually and check that they interact correctly. This last part is the topic of this article. The usual way to solve this problem is to use mocks and spies to prove the correctness of the interactions. This solves the correctness problem quite neatly, but has the undesirable side effect of making the tests brittle.

An alternative is to use property-based testing to verify that components integrate correctly. This isn't like proof, it's a matter of numbers. Throw enough random test cases at the system, and you'll be confident that it works. How many? Enough.

You can get more practical testing skills within the framework of practical online courses from industry experts.