comparison of collections without taking into account the order
While working on a Django Rest Framework (DRF) project, I encountered the need to write tests for an API that returned unsorted data. Sorting data in the API was not required, and doing it just for the sake of tests seemed illogical. Using sets to solve this problem turned out to be impossible, since the elements of a set must be hashable, which dictionaries are not. I was looking for a built-in way to compare unsorted data in pytest, but I did not find any such means. But I came across discussion in the pytest community, where users asked for such a feature to be implemented, and pytest developers suggested that someone else do it as a plugin. This is how the idea of creating pytest-unordered.
Sets
At first glance, the use of sets (set
) seems like a natural solution for such problems. However, this approach has several significant limitations:
Inability to work with non-hashable elements:
Sets require that elements be hashable, which makes them impossible to use with elements such as lists or dictionaries.
Trying to convert collections with complex data structures into sets results in errors and the need for additional conversions.
Loss of structure information:
Inability to compare nested structures:
Using sets does not work for nested structures such as lists of dictionaries or complex JSON objects. Comparing such structures requires a more flexible approach.
assertCountEqual
For comparing unordered collections, the Python standard module library also provides a method unittest.TestCase.assertCountEqual
. This method checks that two lists contain the same number of occurrences of each element, regardless of their order:
import unittest
def test_list_equality(self):
actual = [3, 1, 2, 2]
expected = [1, 2, 3, 2]
assert unittest.TestCase().assertCountEqual(actual, expected)
However, unittest.TestCase.assertCountEqual
has its drawbacks:
Not pretty:
Inability to compare complex data structures:
The pytest-unordered approach
To solve the above problems, pytest-unordered uses an approach similar to that used in pytest.approx
: function unordered
creates an object that overrides the comparison method __eq__
This makes the following possible:
Support for complex data structures:
pytest-unordered
allows you to compare lists, tuples, and even nested data structures such as lists of dictionaries without having to convert them to sets. Simply wrap the desired elements of the structure inunordered()
.Saving duplicates:
Unlike sets,pytest-unordered
Correctly handles cases where collections may contain duplicate elements, preserving their number.Simplifying test code:
Usagepytest-unordered
makes tests more readable and understandable. No need to worry about pre-sorting or transforming data before comparing it.
Features and usage examples of pytest-unordered
Let's look at some usage examples pytest-unordered
.
First you need to install the package using pip
:
pip install pytest-unordered
Comparing lists without regard to order
Let's look at a simple example of comparing lists:
from pytest_unordered import unordered
def test_list_equality():
actual = [3, 1, 2]
expected = [1, 2, 3]
assert actual == unordered(expected)
Here unordered
allows you to check that two lists contain the same elements, regardless of their order.
Comparison of dictionary lists
pytest-unordered
also supports comparison of dictionary lists:
def test_lists_of_dicts():
actual = [
{"name": "Alice", "age": 30},
{"name": "Bob", "age": 25}
]
expected = unordered([
{"name": "Bob", "age": 25},
{"name": "Alice", "age": 30}
])
assert actual == expected
This test checks that both lists contain the same dictionaries, regardless of their order.
Complex data structures
pytest-unordered
allows individual collections within complex structures to be marked as unordered.
def test_nested():
expected = unordered([
{"customer": "Alice", "orders": unordered([123, 456])},
{"customer": "Bob", "orders": [789, 1000]},
])
actual = [
{"customer": "Bob", "orders": [789, 1000]},
{"customer": "Alice", "orders": [456, 123]},
]
assert actual == expected
Here, the external customer lists as well as Alice's orders are checked without regard to the order of the elements, while Bob's orders are checked with regard to the order.
Working with duplicates
pytest-unordered
correctly handles cases where collections contain duplicates:
def test_with_duplicates():
actual = [1, 2, 2, 3]
expected = [3, 2, 1]
assert actual == unordered(expected)
def test_with_duplicates():
actual = [1, 2, 2, 3]
expected = [3, 2, 1]
> assert actual == unordered(expected)
E assert [1, 2, 2, 3] == [3, 2, 1]
E Extra items in the left sequence:
E 2
It is easy to see that using sets to compare these lists would produce a different result.
Checking collection types
If a single argument is passed to the unordered function as a collection, a check will be performed to ensure that the collection types match:
assert [1, 20, 300] == unordered([20, 300, 1])
assert (1, 20, 300) == unordered((20, 300, 1))
If the container types are different, the check will fail:
assert [1, 20, 300] == unordered((20, 300, 1))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
An exception is made for generators:
assert [4, 0, 1] == unordered((i*i for i in range(3)))
To disable type checking, you can pass the elements as separate arguments:
assert [1, 20, 300] == unordered(20, 300, 1)
assert (1, 20, 300) == unordered(20, 300, 1)
You can also specify the parameter explicitly. check_type
:
assert [1, 20, 300] == unordered((20, 300, 1), check_type=False)
What does pytest have to do with it?
The function for comparing collections without regard to order is not specific to pytest
But pytest-unordered
integrates with it through the implementation of a hook pytest_assertrepr_compare
. This allows you to use the plugin's capabilities directly in tests on pytest
and get convenient error messages when checks fail. There was already an example with duplicates above. Here is an example of a message when replacing one element:
def test_unordered():
assert [{"a": 1, "b": 2}, 2, 3] == unordered(2, 3, {"b": 2, "a": 3})
def test_unordered():
> assert [{"a": 1, "b": 2}, 2, 3] == unordered(2, 3, {"b": 2, "a": 3})
E AssertionError: assert [{'a': 1, 'b': 2}, 2, 3] == [2, 3, {'b': 2, 'a': 3}]
E One item replaced:
E Common items:
E {'b': 2}
E Differing items:
E {'a': 1} != {'a': 3}
E
E Full diff:
E {
E - 'a': 3,
E ? ^
E + 'a': 1,
E ? ^
E 'b': 2,
E }
Implementation of the comparison algorithm
The key part pytest-unordered
– this is class UnorderedList
which implements the logic of comparing collections without taking into account the order in the method compare_to
.
The compare_to method code
def compare_to(self, other: List) -> Tuple[List, List]:
extra_left = list(self)
extra_right = []
reordered = []
placeholder = object()
for elem in other:
try:
extra_left.remove(elem)
reordered.append(elem)
except ValueError:
extra_right.append(elem)
reordered.append(placeholder)
placeholder_fillers = extra_left.copy()
for i, elem in reversed(list(enumerate(reordered))):
if not placeholder_fillers:
break
if elem == placeholder:
reordered[i] = placeholder_fillers.pop()
self[:] = [e for e in reordered if e is not placeholder]
return extra_left, extra_right
Method compare_to
performs the actual comparison of collection elements. Initially, all elements self
are copied to extra_left
. The elements of the compared list are checked for presence in extra_left
. Found elements are removed from extra_left
and the missing ones are added to extra_right
. If all elements are found, they are placed in reordered
in the correct order. Placeholders are used for missing elements, which are then replaced by the elements of the comparison list in the order in which they were encountered. The remaining elements are then returned from extra_left
And extra_right
.
Reordering elements in compare_to
is used to create a visual display of data when comparing it visually in the development environment. If elements are out of place, using placeholders helps to determine the exact positions of missing and erroneous elements. This greatly improves the readability of error messages in the IDE and makes it easier to debug tests. Here is an example:
def test_reordering():
expected = unordered([
{"customer": "Charlie", "orders": [123, 456]},
{"customer": "Alice", "orders": unordered([123, 456])},
{"customer": "Bob", "orders": [789, 1000]},
])
actual = [
{"customer": "Alice", "orders": [456, 123]},
{"customer": "Bob", "orders": [789, 1000]},
{"customer": "Charles", "orders": [123, 456]},
]
assert actual == expected
![Without reordering Without reordering](https://habrastorage.org/getpro/habr/upload_files/b72/c40/a2f/b72c40a2fd32ec2a000179a524fa6409.png)
![After reordering After reordering](https://habrastorage.org/getpro/habr/upload_files/53d/890/8dc/53d8908dc383618349be8405ff6b5127.png)
Advantages of pytest-unordered
pytest-unordered
provides many advantages over using sets or pre-sorting data:
The plugin eliminates the need for additional data transformations and makes tests simpler and clearer.
pytest-unordered
allows you to easily compare complex and nested data structures without any additional effort.Unlike sets,
pytest-unordered
handles duplicates correctly.Test code using
unordered
becomes more readable and easier to maintain because it reflects the true intent of the test – to compare a set of elements, ignoring their order.Reordering elements makes visual comparison of different collections more intuitive in the development environment.
Conclusion
If you are working with data whose order does not matter, try pytest-unordered
in your projects. This will help you write simpler, more efficient, and more understandable tests.
At the time of writing repository pytest-unordered
collected 40 stars on GitHub. In addition to me, other people took part in the development three personsfor which I am very grateful. I invite interested members of the community to discuss the project and take part in its development.