Who is responsible for the quality of software?

6 min


Agile software development methodology and DevOps, and especially their emphasis on user experience, draws our attention to the people behind the products. But does the development process really matter, or does the end simply justify the means? London P3X or People, Product, and Process Exchange is heavily focused on the intersection of these three Ps, with the latter being perhaps the most interesting as it combines the largest number of acronyms – test-driven development (TDD), behavior-driven development (BDD – behavior-driven development), continuous delivery (CD), domain-driven development (DDD) and many others to help teams determine how to systematically build better systems.

Co-founder of Agile Testing Fellowship Janet Gregory concluded her keynote address on the software quality assurance process by asking the audience to raise their hands if they feel a moment of magic in their agile team and feel like they are bringing quality to the world. Only a handful of people from the audience, supposedly all of them practicing agile developers, raised their hands.

How did it happen that since the signing Agile manifesto 17 years have passed (at the time of the report), and still so few people have gone beyond contemplation shadows on the cave wall? Maybe we still don’t have the right dialogue. Maybe we still don’t have it with the right people. Or maybe dialogues are not part of our processes at all.

While the manifesto places people and their interactions above processes and tools, there is still something inherently human in processes. Perhaps by learning about our processes, we can better respond to change, expand our collaboration, and reduce errors – all for timely and consistent customer satisfaction. Gregory took an approach to quality that we have known for generations and applied it to today’s agile software development teams in the hope that everyone would become a co-owner of what he releases.

But what is “quality”?

Gregory noted that the subjective definition of quality often needs to be predetermined. To start with the definition of quality somewhere, she suggested Five Approaches to Determining Quality by David A. Garvin 1984:

  • Transcendental (abstract) – Atmosphere, innate excellence, universally recognizable virtues

  • Value Based – Price and Cost

  • From a User Perspective – User value is what most people will represent when thinking about quality

  • Product Based – What do your users need? (for example, what kind of milk is presented)

  • Manufacturing-Based – Methods, Processes, Standards, Requirements, Specifications – Have we set up production correctly?

Gregory visualized the relevance of each category in the following way, where the most pressing ones are closer to the center, and applied this to the modern agile environment.

Manufacturing approach

First, everything has to work, so production quality is at the core.

Gregory argues that it involves test-driven design because “writing clean code dramatically reduces rework. Let’s do it right the first time, so as not to produce bugs. Then we can release it with confidence. “

TDD, the practice of developing automated tests down to the software under test itself, which in turn reduces the coherence of that software, is an important part of manufacturing quality. Gregory cited a study that says teams that implement TDD end up with 60-90 percent fewer defects than those who don’t, but TDD takes an average of 15-30 percent more development time.

Many teams are faced with this trade-off between quality and speed.

“It may happen that the PO (Product Owner) says that they would rather implement a new feature than quality. Who should make such decisions? “

In addition to TDD, Gregory revealed that manufacturing processes include:

  • Writing maintainability code

  • Monitoring error logs

  • Continuous integration

  • Exploratory testing by stories

  • Testing the product against specifications

  • Automatically generate tests for quick feedback

  • Static platform analysis

  • Clear definition of “Done” status

Finally, she said, “Practitioners like DevOps try to reduce customer risk during release to their customers.”

Product-based approach

Simply put, if production quality is about making something work, then product quality is about making sure it works as it should. For example, we tend to pay more for higher quality, and we are more lenient if something that costs very little breaks. An exception may be free apps, which we expect to work well.

Gregory points out that what is defined as product quality depends on the target audience. Some accountants may desperately need a keyboard pad, which is no longer available in most laptop computers today.

The whole point is in two questions:

  • Are we creating what is needed?

  • Are we adding functionality that we need?

This includes:

  • Acceptance Test Driven Development (ATDD) or, as it is sometimes called, story driven development – bringing key customers to the TDD phase

  • Security testing

  • Eliminating bugs, such as team hackathon to find as many bugs as possible

  • Continuous delivery

  • Exploratory feature testing

  • Beta version

  • Performance testing

  • Stress Testing

User perspective

This is where points of view differ. As Gregory said, “Different people choose different things. They have different desires, different needs. If we are trying to give the buyer a choice, we still need to satisfy him. “

But don’t forget, she continued, “we’re also making the assumption that consumers have enough information to make an informed decision.”

She spoke about an app she once used, which she found very unfriendly. It turned out that users liked it because it exactly matched their principles of work. And she did not work in this area. It’s all about satisfying specific use cases for specific users.

Value-based approach

It’s just what people are willing to pay for. It is difficult to judge the value, almost impossible without knowing the opinion of potential customers.

Quality in terms of value is assessed using some combination of the following factors:

  • Efficiency

  • Practicality

  • Comfort

  • The trust

  • Complexity

  • Matching the context

Transcendental quality

And, finally, the most difficult quality to measure is transcendental. Gregory explains that this is because emotions are difficult to measure, and transcendental quality comes from a combination of artistry, engagement and customer loyalty.

How do we measure software quality?

In general, if you agree with the Garvin quality scale, a significant proportion of software quality is difficult to measure. She quoted Isabelle Evans regarding software quality assessment. There are many examples of manufacturing quality:

  • Number of production bugs

  • The severity of production errors

  • Number of days since the last launch in production

  • Number of new support calls in X days since the last release

  • Build indicator stays green

  • No failed automated tests (random failures)

  • Static analysis of the code base does not reveal problems

  • Low-level fixes

  • The same bugs don’t pop up again

There is also a measure of quality from the user’s point of view in the form of user satisfaction surveys.

However, you cannot really measure quality in terms of product, value, or transcendental quality. However, you can discuss and evaluate all five levels of quality. Testing is an important indicator of quality, but Gregory reminds us that product groups cannot deny the value of discussing quality with each other, with users and with competitors.

And of course, teams need to strike a balance between the desire to avoid mistakes and the speed of development.

The whole team is responsible for quality

Obviously, quality assurance – or trying to solve this misnamed impossibility – is not just a task for one testing department, to which the code is simply pushed.

The whole team owns quality

“If your organization, if your company starts out with a quality concern, you will probably succeed because everything else will fall into place. Everything will work naturally. But if you start with the idea that speed is the most important thing, regardless of quality, chances are that in the long term you will have to do a lot of rework, fix a lot of unsupported code, and your quality will decline further and further, ”says Gregory.

But she hasn’t revealed any secret recipe for the perfect pursuit of quality.

“What quality approaches you use I don’t care, but you have to ask yourself when you look at your processes – does that lead to confidence in releases?” says Gregory. “Does the quality of the process measure the quality of the product?”

She quoted a co-author BDD Book 1: Discovery Seba Rose: “When a measure becomes a goal, it stops being a good measure.”

“No matter how you measure it, it should lead to a conversation, a discussion about what you need,” says Gregory.

She continued, “The team has quality, but you need to think bigger. Quality in processes and quality in practice. Competence in your team, competence in how you deliver software. “

In the end, she said that any dialogue in this direction is best started with people trying to solve a problem.

“Let’s build quality management into our process and learn to talk about what we do,” said Gregory.


The translation of the article was prepared specially in anticipation of the start of the course “QA Lead”… And we invite everyone to free demo lesson of the course on the topic: “Organization of testing with different development methodologies”


0 Comments

Leave a Reply