what they will tell about testing at Heisenbug 2023 Autumn

The art of JUnit extensions

Dmitry Tuchs

This is not just a report, but a workshop. Let’s take a microservice project online (Spring Authorization Server, Spring OAuth 2.0 Resource Server) and build a complete infrastructure for working with test data, authorization, and others using JUnit Extensions. Only practical skills that will be useful to most Java automators.


Model-Based Testing with Hypothesis

Evgeny Zayats

Eugene will consider the application of the model-based testing approach using the PyTest and hypothesis bundle. Let’s talk about the basic concepts, finite automata, delve into the details of using the hypothesis and try to solve some interesting problems with it. We will also learn when to use this approach is not worth it and what rake can be avoided. If you are looking for new ways to diversify testing in your project, come in, it will be interesting.


Release Quality Assessment

Daria Melnikova

Credit Bank of Moscow

The report addresses the issues of version release quality, as well as advance planning and system management. Daria will talk about how the released changes were evaluated based on the number of incidents, approvals, planning results and customer positions.

It will be interesting for teams that have release regulations or integration tasks between teams and / or third-party resources.


Variety of mock tools

Aleksandr Kuznetsov

MTS Bank

Alexander will analyze the existing ones and analyze popular request mocking tools:

Mockoon;
Prism;
Castle Mock and others.

Consider the main features and benefits of each of the tools.


Development of visualization tools for analyzing coverage of auto API tests

Kirill Khrabrov

X5 Technology

Kirill will consider topical issues related to the definition and visualization of test coverage by auto API tests. The focus will be on the information source selection mechanism, which determines the target coverage percentage of auto API tests and provides information about the actual coverage.

Various options for visualization of analytical information about test coverage will be presented, based on the use of a technology stack that includes Kibana, Allure, Python and GitLab.


How we built in fuzzing testing, or How to hit a bunch of problems with one commit

Ruslan Kosykh

Ozon Fintech

Ruslan will tell you how they screwed up fuzzing tests and what it led to.

He will start with the most interesting – why they needed it. He will talk about choosing a tool and ways to connect fuzzing tests, what problems they encountered in the process, how they were solved. And in the end, look at what problems have been identified.

The report will be of interest to those who are in one way or another connected with the backend:

developers who are looking for problem areas without involving a testing team;
QA, who just came to the project, will give an idea how to suddenly cover the API backend with a large number of cases;
QA-old-timers of the project will see a case of real application of fuzzing tests.


Distributed running of E2E tests in CI

Roman Zolotarev

Yclients

The long passage of E2E tests is a known pain for many. Roman will share his experience of how they relieved this pain. He will talk about:

The evolution of the approach to running E2E tests in CI.
Experience in writing your own tool for distributed test execution using Python and PyTest as an example.
Transition from vertical to horizontal scaling of resources required for tests.
The possibility of obtaining a conditionally infinite number of test execution threads.


How we optimized e2e launches and re-optimized

Alexey Romanov

Ozon Fintech

The speaker’s team was faced with the task of learning how to run e2e API tests for a specific service without pain and unnecessary tests. In the report, Alexey will share how they got to launch tests on traces, what problems they have with this, and how it helped them speed up pipelines.

The report will use pytest + xdist. The optimization service was written in Go, but we will only consider it schematically, without reference to the language. The speaker will also tell how they laid out the tests by workers and accelerated the specific launch of the pipeline.


Test Automation Platform Cloud Device Farm

Evgeny Shkalev

VK

Evgeny will talk about the device farm (browsers/emulators) for running tests in parallel. He will show an example of their system built using available open source tools, libraries and technologies. The speaker will also talk about moving the farm from hardware to the cloud: what it cost for them and what benefits it brought. Issue: Running a large number of tests that require devices (web browsers and Android emulators).

Technologies: Java, Selenium Standalone Server, Android SDK Tools.

The report will be of interest to SDET, automators and QA engineers.


Best practices

The Future is Today: Leveraging AI in Software Testing

Mesut Durukal

Indeed

Mesut will talk about the effective use of machine learning in software testing, give some case studies and share the results of a case study that he used on his project to debug bugs.
Testing is a very labor intensive process. An agile approach means that expected behavior can change over time. In addition, after making changes, tests can be broken. And time is limited! Minimize manual testing to increase coverage and allocate time for research with limited resources.
Machine learning allows you to minimize manual testing and make it more convenient and consistent. Testing can be applied at the following stages:

  • Definition of testing.

  • Automatic code generation.

  • Implementation: exploratory testing.

  • maintenance and grouping.

  • Test code review.

  • Correction of the test code.

  • Prioritization of test cases.

  • Debugging bugs.

We will see how machine learning helps at all stages. In the presentation, the speaker will list all areas of application with algorithms and discuss the arguments for the use of AI and the risks for software testing.


If you want 400 releases a week, use these engineering practices

Egor Struchin

Ozon Fintech

The experience of automating Go-microservices and the work of the SDET team contains many different topics. Egor will talk about what has not yet been considered in a large audience, and where achievements are obtained for a long time and interestingly – we will discuss the generation, updating and reuse of test data; what to pay attention to when covering with tests and why; how to help developers participate in QA processes for their own good, as well as how to make development and testing friends with the help of unit tests.


How we automated tests with multiple users

Vasily Vanchuk

At a food delivery service, Vasily and his team faced the challenge of writing E2E tests. The main challenge was that in order to test the entities of the client, courier and restaurant, you need to act on behalf of other entities – for the restaurant to be online, the couriers to be in the city, etc.

At first they tried the approach with mocks and a fake API, but this did not suit the team, because only the mobile application was tested and in artificial conditions. Later, they created (and successfully) a library that, through API calls, allows you to simulate all the actors in the system.

The speaker will talk about how the decision was made and how it all ended.


Load testing in Yandex. What if developers test

Alexander Ivanov

Yandex

The speaker will talk about how load testing works in Yandex, demonstrate the results of the active involvement of developers in the process, and share the pros and cons of this approach.

Let’s discuss what the applied practices can give companies, developers and load testers. In addition, Alexander will show how you can save time on the basis of Yandex tools and do the same for yourself.


Antipatterns in automation and how Playwright avoids them

Edip Asanov

Ozon Fintech

Choosing a powerful tool is not a guarantee of success: you still need to know how to use it. It also turns out that gaining experience is largely about understanding what practices and decisions to avoid.

In the report, we will talk about typical mistakes when writing autotests, which lead to the degradation of the code base, an increase in the cost of support, and even the inoperability of the tests themselves. We will focus on how typical automation problems that lead the tester to the path of crutches were solved in Playwright.


API layer in functional tests: subtleties of customization

Julia Atlasova

Signant Health

How to add an API layer to functional tests? The API is a great help in UI tests and tests for checking the business logic of applications. But how to competently embed the API layer so that it is universal, so that it can be easily reused, modified, applied in combination with UI steps?

Julia will talk about the technical intricacies of working with the API layer and show everything with specific examples. She will also talk about builders, code grouping using a ServiceObject (PageObject analogue for API services), general connection settings, request ID logging, and other techniques.


Web Application Performance: Myths and Facts About Web Vitals

Aleksey Ivanov

Airplane

Web Vitals is becoming an important tool for evaluating the performance of web applications, but there are also some doubts about its reliability and applicability. During the report, we will analyze the key aspects related to Web Vitals and identify possible limitations that should be considered when using this metric. We will also discuss additional tools that can be useful for a more complete analysis of the performance of a web application.


Metrics in testing, or how big brother is watching you

Alexander Petrov

VKontakte for business

Based on the experience of building metrics in Citymobil, Yandex, SberZvuk, VK, Alexander will talk about what metrics are useful, what his team did and made mistakes in the value of these metrics, what knowledge can be learned from them and how managers use them. There will also be some funny cases from life related to testing metrics.


Mobile testing

Integration testing on Flutter and CI/CD

Akhmat Sultanov

AVO.uz

As part of the report, we will dive into the analysis of the features of UI testing using integration tests on Flutter. The speaker will identify and discuss in detail the key issues that affect the time spent in conducting such tests, as well as present effective solutions.

Akhmat will take a close look at dynamic creation of emulators to run CI tests in parallel to optimize test times. The purpose of the analysis is to demonstrate improvements in the efficiency and speed of CI/CD processes.

The report will be useful for specialists in the field of test automation (SDET) and developers who want to explore in more depth the possibilities and nuances of testing using Flutter.


We reduce the time of UI tests on iOS from 4 hours to 30 minutes

Boris Lysikov

Sound

At the beginning of the year, Boris’s team was given an ambitious goal to reduce the time it takes to pass UI tests on iOS. During the report, Boris will share the best practices that were used to achieve the goal and talk about how the team managed to reduce the time for passing tests by 8 times and at the same time not lose stability.


How to tame an Android emulator

Ivan Levikov

In contact with

Ivan will talk about how to assemble and prepare Android emulators for running autotests with your own hands. The speaker will explain what the various parameters in the dockerfile mean, why they are needed and how to improve the build of emulators by automating manual steps. We will also discuss how to create and run a fast emulator and, if necessary, save the changes made.


Embedded, Mobile, Web and how it all connects in Flipper Zero

Konstantin Volkov

Flipper Devices

We have all come across embedded devices – these can be weather stations, controllers in our car, even a regular intercom. The level of integration of such devices is constantly growing.

We are used to the fact that we can interact with a variety of devices through a smartphone or browser. Flipper Zero is no exception: if we want to update the firmware of the device, backup keys or remotely work with the device, then we can use one of the companions.

Have you worked with Web Serial, Appium, Selenium or just want to know how it can be used with Flipper? We tried to combine all this and use it in testing our device. Let’s discuss building all testing processes from scratch.


This you have not seen before. How to automate a WebView in a hybrid mobile app without changing jobs

Andrey Popov

X5 Technology

Elvina Grange

X5 Technology

The hybrid architecture of the mobile application provides both opportunities and limitations in testing.

What problems can be encountered and how to solve them? Speakers will tell you what tools they chose to automate the web part, how they integrated the entire QA team into writing E2E tests, and how 70% of the regression was automated in a short time.


Hardware

Autonomous Vehicle Testing: Where Software and Hardware QA Meet

Dmitry Nikitin

Evocargo

Evocargo produces its own autonomous vehicles for the transport of goods. Each vehicle that leaves the assembly line is subjected to a number of tests. Some of them are field tests. Dmitry will tell about them in the report.

The speaker will show how they test their cars when they are released from production, as well as how they decide whether a new development is ready to go into serial construction. And all this with examples from real practice.


Challenges of IoT Software Testing

Sergei Shaikin

Electrolux AB

Sergey will briefly walk through IoT software testing, analyze the importance of testing for functionality, security and reliability, as well as consider the difficulties and ways to solve them.

Sergey will give practical advice to testers who want to move into IoT software testing: what to expect, how best to prepare and where to practice.

Let’s discuss possible ways of further development of the Internet of things in general and its testing in particular.


Testing software and hardware solutions within SSDL

Svetlana Korneeva

Aladdin

Anton Bautkin

Aladdin

Report on best practices for testing smart card devices using USB tokens as an example. The purpose of the report is to consider the features of testing USB tokens, namely, complex testing of the hardware component of the media, firmware and application programs for managing the media. Testing methods will also be considered using the following instruments: a pyrometer and a thermal imager, a USB tester and a specialized network relay module when examining such product performance characteristics as media operating temperature, current consumption, microSD card speed in media, and media behavior under load.

The report will present a brief theory of what USB tokens are and why they are needed, as well as describe the methods and results of testing USB tokens. As a result, the report reflects the variety of tasks that modern testing specialists face.

The report will be useful for testing specialists who are involved in testing hardware media or other equipment. The level of specialists is middle and higher.


Stress Testing

Examples of atypical scenarios in PostgreSQL load testing

Mikhail Zhilin

PostgreSQL

PostgreSQL is a popular OLTP database that allows reaching millions of transactions per second on SELECT/DML queries. During the report, the speaker will talk about atypical loads, give examples and analyze them using profiling. Let’s discuss why COMMIT can slow down and be to blame, what are the pitfalls when creating one million tables, how you can quickly connect and why it can’t be faster, and also consider an example of a query that starts to “stupid” when there is contention.


The show must go on: how to load a conference to keep 10 thousand participants

Vladimir Dyer

JUG Group

Since 2019, JUG Ru Group has been developing a solution for online conferences. The team also began to use the same solution for meeting external customers. One of the requirements for the solution is scalability and fault tolerance: no matter what, the solution must “hold” a conference with 10K online participants. The show must go on!

Vladimir will talk about the main components of the solution, why a bunch of gatling and nanocloud was chosen as a load testing tool, how the load profile was built and what optimizations were carried out in the solution components and infrastructure to get closer to the coveted 10K.


pg_profile – PostgreSQL strategic monitoring utility

Andrey Zubkov

PostgreSQL

The main task of strategic monitoring is to provide quantitative information about the most resource-intensive DBMS activities. Such information is very useful for DBMS load analysis, for example, when performing load testing.

Strategic monitoring by itself does not solve any problems, but it can suggest what to pay attention to. One strategic monitoring tool is the pg_profile/pgpro_pwr extension. The speaker will talk about the architecture, capabilities, features of work and how to use these extensions.

The report will be of interest to all those who work with the Postgres DBMS – database administrators, load testing engineers and developers of applications that directly interact with Postgres databases.


Safety

Not All Mistakes Are Equal

Anna Vasilyeva

ATI.SU

Entered an invalid value in the API – received an error text or an unhandled exception. The error text helps users enter and submit the correct values, and QA checks this in autotests.

But not all errors are equally useful from a security point of view:

Some error texts may reveal information about the system and internal infrastructure.
Validation may contain vulnerabilities.
Errors can be confusing for the tester – when, focusing on the error, we believe that the method has not been executed. Sometimes it doesn’t.
During the presentation, we will consider examples where a “bug” is a security issue. The report will be of interest to anyone interested in security testing.


space baghunting

Andrey Moiseev

InfoTeKS

Alexey Fedulaev

Wildberries

Space is a little-studied, unpredictable and aggressive environment. Have you ever wondered how to test spacecraft? What’s more, the price of a mistake here is made absolute: a failed launch can cost billions of dollars and decades of development.

What dangers can a spacecraft face in space and how can they affect the operation of the device? What protection mechanisms are needed to compensate for them? How to test all this on the ground?

We will talk about this and more in the report.


UI

Why Yandex Business Trips no longer write screenshot tests

Valery Baranov

Yandex

Valery will talk about the evolution of screenshot tests in Yandex Business Trips and why they switched from writing tests manually to generating them at runtime.

Let’s take a look at the details of the transition to generating screenshot tests based on storybooks at runtime with the help of Hermione and Playwright. Valery will share the real results of such a transition, as well as the problems they faced and strategies for solving. Let’s discuss the advantages and disadvantages of the approach, which will allow us to make a decision on its application at home.


Features of testing SmartTV applications

Mikhail Gamayunov

Amediateka

What are the features of TV applications? Is it difficult to test SmartTV apps? The speaker will try to show that it is quite simple, given the nuances and the process.

In his report, Mikhail will talk about the two most popular SmartTV platforms on the market. Let’s discuss the features of different types of testing: functional, UI, UX. Let’s talk about the organization of the testing process, the tools used, as well as the difficulties of delivering builds to the device.

The report will be useful both for those who want to get acquainted with the testing of SmartTV applications, and for those who have at least once asked the question: “What about experienced QA from the world of TVs?”


Kinotes

What prevents us from making a quality product?

Vitaly Sharovatov

Are you proud of the product you make? Do you sincerely recommend it to friends and family? Are you glad to invite friends to work for your company? Is your product a pleasure to use? Do you love your work? What is your influence on the product being created?

We are all managers. Engineers manage only their own work, leaders manage teams. One of the tasks of management is to ensure the proper quality of the product.

Why are there so many bad products on the market? Why is there so much shame and self-justification among employees?

This is due to a lot of irrational approaches and practices: from the separation of QA and programmers to time tracking and sprints, from bonuses to hyper-specialization, from performance reviews to code reviews, from “we don’t write tests” to “I did, QA will check.”

In the report, the speaker will consider the most common and most harmful of these approaches and practices and suggest better alternatives.


The quantum structure of the world: from the Nobel Prize 2022 to experiencing reality inside a quantum computer

Alexey Semikhatov

Physical Institute. P.N. Lebedev RAS

Quantum mechanics is the most amazing thing that has ever been invented by people: it makes successful predictions about unimaginable objects, but does not explain how these predictions come true and “what happens”.

Theoretical and experimental efforts to elucidate the structure of quantum reality demonstrate its break with the world we are accustomed to. Possible interpretations range from parallel universes to agent-centric cubism. High-tech experiments indicate a loss of local realism, and a significant next step in understanding quantum nature would be high-level AI (“Wigner’s friend”) implemented in a quantum computer.

Alexey will tell you what terms quantum mechanics uses, what conclusions about the structure of reality (prohibitions on habitual properties) can be drawn from it logically, what was experimentally verified (Nobel Prize 2022), what trends (including friendliness) have been discussed lately, and why the key expectations are now associated with agents operating inside quantum computers.


AI

UI recognition in medicine

Mikhail Zuev

Auriga

Dmitry Volochaev

Auriga

Modern medicine uses high-tech equipment to monitor the patient’s condition in situations with a high risk to life. The reliability of such equipment is also ensured by the software testing team.

Speakers will talk about approaches to automating GUI testing using artificial intelligence elements. Consider the possibilities of screen capture recognition, the risks associated with this approach, as well as methods that allow you to control these risks. Particular emphasis is placed on the recognition of text, icons and graphics.

The report will be useful to those who are interested in the use of image recognition in testing and who at least once lay under a medical monitor and asked the question “How is it tested?”


Finally

Of course, the conference is not only reports. Many participants directly say that communication is even more important for them: if you missed an interesting report, then you can at least watch it in the recording, but you cannot convey the missed communication with a recording.

And we try to ensure that there are the most suitable conditions for this. For example, after each report, we have not just “three minutes for questions”, but a whole “discussion zone” where you can ask the speaker in detail – sometimes there are heated discussions for an entire hour. This works both offline (where a special place is provided for the discussion zone), and online (where questions are asked in the format of a video call).

And, of course, there is simply a lot of communication between participants offline. But you can’t convey all this with a habrapost: about reports, we can write “these are the topics waiting for you,” but in the case of communication, everything is in your hands. So it only remains for us to finally recall the main thing: October 10-11 – the online part, October 15-16 in St. Petersburg (with the possibility of online connection), all other details and tickets – for conference website.

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *