Qualification of tools for embedded software development

Hello habr! In this article, I want to tell you in the most simple and accessible way about how it is proved that your development and verification tools are suitable for creating systems with increased reliability. This is a very important and far from the easiest question, and my goal is to answer it in the most understandable language possible. In the article itself, I summarized guidelines from industry standards such as KT-178 or R-331 (embedded software in aviation), GOST R ISO 26262-8 (embedded software in the automotive industry). So welcome under the cut

Qualification – why is it?

Industry standards such as KT-178 or ISO 26262 describe processes for building robust software. If you follow these descriptions, the creation of such software will turn into a bureaucratic hell that will last forever. But there is software that can automate a significant part of these processes. Such software is called a tool. And if you use a tool, then it must be reliable (ISO 26262 even introduces the term “confidence in the tool”). To confirm the reliability of the instrument, its qualification is carried out.

Qualification – theory

Industry standards have a concept of security level. In different standards, they are called differently: Software Level in CT-178, Vehicle Safety Completeness Levels in ISO 26262. And for development tools, tool qualification levels (CT-178) or tool classification levels (ISO 26262) are used. These levels are assigned based on the criticality of the tools – the more influence a tool has on development, the higher the skill level will be assigned to it. In this case, one of the main criteria for determining the influence of a tool is the measure of its influence on the resulting software.
Consider a source code generator and a static code analyzer as examples. The generated code goes into the firmware of the device, which will be installed on board an aircraft or car. Thus, the code generator has a direct impact on the resulting software. Since a code generator is a complex thing and can generate code with errors, strict requirements are imposed on the quality of this code generator and the level of its qualifications will be maximum. Another thing is a static analyzer, the result of which does not get into the onboard software and the degree of its influence is minimal. Therefore, the skill level for a static analyzer will be lower than for a code generator.
And the level of qualification directly affects labor costs: for example, for aviation, for the qualification of an instrument at the highest level of KT-178S, 76 control measures are required, and for the lowest – only 14.
Another important point – the qualification of tools is carried out not by the developer of the tool, but directly by the software developer, and the qualification must be carried out for each project!

Qualification – Practice Notes

As mentioned in the theoretical part, tool qualification is a costly process, but it is simplified in several ways:

  • Support of the qualification process by tool manufacturers (vendors)
  • Instrument qualification guidelines from standards

Let’s consider them in order. First of all, vendors themselves supply tool qualification support packages. Second, industry standards provide guidance on qualifying tools that can reduce workload.

I would like to dwell on the second point. Let’s look at the code generator again. The code generated with its help must still be tested, the code coverage will be collected, its analysis is carried out, that is, verified.
Industry standards tell us that if we qualify the tools that are responsible for verifying the code, then the qualification of the code generator itself will not be needed. Thus, the standards themselves provide guidance that you can follow to significantly reduce the cost of qualifications.

For the qualification of verification instruments, the standards prescribe demonstrating their behavior under normal conditions. In practice, it looks like this:

  1. The vendor supplies a set of document templates, regulations and benchmarks for the instrument and their reference results.
  2. You fill out document templates and run the provided tests in your environment.
  3. The results of the tests you run are compared with the benchmarks, and if there is a discrepancy in the results, you eliminate the discrepancy.

It is worth noting that the described process also takes a lot of time and, like all development, is carried out in cooperation with the certifying authorities.

MathWorks tools and their qualifications

Tools such as Simulink, DSP Toolbox, Control System Toolbox are the industry standard for developing control systems, digital signal processing. It is not surprising that they are used in aviation, automotive and other industries. The developed models are used to generate C / C ++ code that travels and flies. Naturally, developers are faced with the question of tool qualifications. And the qualification of MathWorks tools for KT-178S is carried out for model and code verification tools:

And for ISO 26262, certificates are supplied for:

  • Simulink Check
  • Simulink Coverage
  • Simulink Requirements
  • Simulink Design Verifier
  • Simulink Test
  • Simulink Report Generator
  • Polyspace Bug Finder
  • Polyspace Code Prover
  • Embedded Coder
  • HDL Coder
  • PLC Coder

Industry-specific tool qualification support packages are provided DO Qualification Kit for aviation or IEC Certification Kit for the automotive, railway and other industries.

Instead of conclusions

This article has provided a high-level overview of the industry-standard tool qualification process. In the course of writing this article, I was guided by the following sources:

  1. R-330, “Software Tools Qualification Guide”, in particular:

  • p. 2.0 Assignment of tool qualifications
  • p. 3.1. Qualification levels
  • Reference D question D7

  1. GOST R ISO 26262-8, Chapter 11, “Confidence in using tooling software”

If you are interested in how systems of increased reliability are created, then write in the comments what you need to consider in the following articles.

Similar Posts

Leave a Reply