Hardpy. Nucleo-f401 example — automating electronics testing in production using Python

How not to waste time on bicycles, but simply write tests in python and enjoy stable working test stations in electronics production? Of course, using an open source solution HardPy.

Analysis of the use of functions HardPy – an open framework for creating test stations for electronics production in Python using the example of testing and flashing the Nucleo-F401 debug board.

Why is this necessary? When an electronic device is finally developed and successfully tested for compliance with all requirements, it is time to launch production. With a large number of functions, large batches or simply high quality requirements, the task of automating functional testing and firmware of boards in production arises. Writing and debugging such software from scratch sometimes took more resources from us than developing the product itself.

Let's look at a simple example of how to do this quickly and reliably using HardPy. The goal is to show the basic steps of developing software for testing.

Test stand

In order to build a test bench, we need to answer several questions:

• what device are we testing – (DUT – Device under test)

• what we want to test – list of functions

• how we will test it – test plan

• what we test it with – devices, structural diagram of the stand

DUT

The device under test is a debug board. Nucleo-F401RE . Nucleo – ascetic debugging, which offers a minimum of functions at a cool price. A third of the board is occupied by the ST-LINK programmer, we will consider it a device, and not part of the tested board.

We determine the scope of testing

What functional blocks do we want to test on this board:

To simulate the real world, we will introduce a defect simulator on the board – jumper. It will allow you to create (there is no defect) or break the connection (there is a defect) between the microcontroller pins.

We are testing a small batch, so we will not automate the detection of the LED operation and the activation of the button, we will use the eyes and hands of the tester for this. To reduce the influence of the human factor, we will vary the tasks and check their execution by a person together with the DUT.

Now that we know exactly what we are testing and in what volume, we can begin to develop a test plan.

Test plan

1. Check the availability of devices;

2. Check for firmware availability for DUT;

3. Flash DUT;

4. Read DUT serial number;

5. Check for the presence of a jumper;

6. Check the LED, blink the LED several times, ask the user the number of blinks;

7. Check User Button, ask the user to click the button an arbitrary number of times (variation) on the button. We ask to enter this number in the form.

Now we know what we are testing and how. Let's decide on the composition of the test stand.

Test station calculator

Any computer with Linux. You can also run the example on Windows, but you shouldn't.

Stand layout

We will record the composition of the stand and all connections between the components on the structural diagram.

To test the DUT we will need only one device – the ST-LINKV2 programmer. Luckily, it is on our Nucleo board, so all that remains is to connect the board to the computer via USB and prepare a jumper

Test station program

We write drivers

Drivers are needed for the program to work with DUT and devices. We have one device and one DUT module.

ST-LINK driver

Luckily, a lot has already been written for us, and we can simply use PyOCD for flashing the STM32 microcontroller using ST-LINKV2.

DUT driver

No one has written ready-made drivers for our DUT, so we write them ourselves.
The driver is quite simple and connects to the device via a serial port to the DUT at a speed of 115200 baud. The driver requests the serial number from the DUT, the status of the presence of the jumper and the number of button presses.

from serial import Serial
from struct import unpack
  

class DutDriver(object):
    def __init__(self, port: str, baud: int) -> None:
	self.serial = self._connect(port, baud)
	self.SERIAL_NUMBER_STR_LEN = 11
	self.SIMPLE_REPLY_LEN = 1

    def write_bytes(self, data: str) -> None:
	self.serial.write(data.encode())

    def read_bytes(self, size: int) -> bytes:
        return self.serial.read(size)

    def reqserial_num(self) -> str:
        self.write_bytes("0")
        reply = self.read_bytes(self.SERIAL_NUMBER_STR_LEN)
        return reply.decode("ascii")[:-1]

    def req_jumper_status(self) -> int:
        self.write_bytes("1")
        reply = self.read_bytes(self.SIMPLE_REPLY_LEN)
        return unpack("<b", reply)[0]

    def req_button_press(self) -> int:
        self.write_bytes("2")
        reply = self.read_bytes(self.SIMPLE_REPLY_LEN)
        return unpack("<b", reply)[0]

    def _connect(self, port: str, baud: int) -> Serial | None:
        try:
            return Serial(port=port, baudrate=baud)
        except IOError as exc:
            print(f"Error open port: {exc}")
            return None

Writing a test plan in pytest

pytest – an open source tool that makes it easy to write small, readable tests and can scale to support complex functional testing of applications and libraries. You can read more about pytest functionality in project documentation

When debugging code, you can conveniently run it directly in your IDE; HardPy is not required for this.

MCU firmware

Let's write a test for DUT firmware. The example will consist of 2 files: conftest.py And test_1_fw.py. In the file conftest.py let's initialize the DUT driver, and in test_1_fw.py the test itself will be there. More about conftest.py you can read it in documentation to pytest. All example files should be in one folder, in our case it will be the folder tests.

conftest.py

IN conftest.py create a driver instance, pass it the port and the speed of 115200 baud (set in DUT). The port may differ on different PCs, so you may need to edit this variable.

# conftest.py

import pytest
from dut_driver import DutDriver

@pytest.fixture(scope="session")
def device_under_test():
    dut = DutDriver("/dev/ttyACM0", 115200)
    yield dut

test_1_fw.py

Testing within a file test_1_fw.py consists of checking the presence of firmware, checking the connection of the DUT to the PC, and flashing the DUT. The compiled DUT firmware must be placed in the folder with the test, dut-nucleo.hexyou can assemble it yourself from sourcesor use it already compiled version.

# test_1_fw.py

from glob import glob
from pathlib import Path

import pytest
from pyocd.core.helpers import ConnectHelper
from pyocd.flash.file_programmer import FileProgrammer

def firmware_find():
    fw_dir = Path(Path(__file__).parent.resolve() / "dut-nucleo.hex")
    return glob(str(fw_dir))

def test_fw_exist():
    assert firmware_find() != []
  
def test_check_connection(device_under_test):
    assert device_under_test.serial is not None, "DUT not connected"

def test_fw_flash():
    fw = firmware_find()[0]
    hardpy.set_message(f"Flashing file {fw}...", "flash_status")

    with ConnectHelper.session_with_chosen_probe(
        target_override="stm32f401retx"
    ) as session:
        # Load firmware into device.
        FileProgrammer(session).program(fw)

To run the tests, we need to install several python packages via pip.

pyserial==3.5
pytest==8.1.1
pyocd==0.36.0

It is recommended to install them in virtual environment venv or conda.

• pyserial used to interact with DUT via serial port

• pyocd used to update DUT firmware via built-in ST-link.

For correct operation pyocd You will also need to install the stm32f401retx support package.
To do this, you need to call the commands:

pyocd pack update
pyocd pack install stm32f401retx

You can run tests from a folder tests command:

pytest .

We receive a message about successful testing:

collected 3 items                                                                                                                                 

test_1_fw.py ...                                                                                                                            [100%]

======= 3 passed in 7.27s =======

Thus, we managed to write a simple test plan that updates the DUT firmware using pytest.

During device development, pytest can be used for test scripts that will check specific scenarios of the device being developed. Ideally, this can be built into the CI/CD, HIL process.

Great, the tests run, but this is not suitable for production work, at a minimum you need an operator panel and storage of test results.

Adding HardPy

Now you can add to tests HardPyinstalling it via pip. (For HardPy version 0.4.0, the python version must be at least 3.10, and the pytest version must be at least 7)

pip install hardpy

Improving our file test_1_fw.py:

# test_1_fw.py

from glob import glob
from pathlib import Path

import hardpy
import pytest

from pyocd.core.helpers import ConnectHelper
from pyocd.flash.file_programmer import FileProgrammer

pytestmark = pytest.mark.module_name("Testing preparation")

def firmware_find():
    fw_dir = Path(Path(__file__).parent.resolve() / "dut-nucleo.hex")
    return glob(str(fw_dir))

@pytest.mark.case_name("Availability of firmware")
def test_fw_exist():
    assert firmware_find() != []

@pytest.mark.case_name("Check DUT connection")  
def test_check_connection(device_under_test):
    assert device_under_test.serial is not None, "DUT not connected"

@pytest.mark.dependency("test_1_fw::test_check_connection")
@pytest.mark.case_name("Flashing firmware")
def test_fw_flash():
    fw = firmware_find()[0]
    hardpy.set_message(f"Flashing file {fw}...", "flash_status")

    with ConnectHelper.session_with_chosen_probe(
        target_override="stm32f401retx"
    ) as session:
        # Load firmware into device.
        FileProgrammer(session).program(fw)

    hardpy.set_message(f"Successfully flashed file {fw}", "flash_status")
    assert True

The following has been added to the file:

• Import package hardpy;

• Messages to the operator, via the function set_message();

• Operator-friendly names of tests and test groups: case_name And module_name in hardpy terminology.

• Dependency marker dependency test test_fw_flash from the test test_check_connection. In case the test test_fw_flash fail, test test_check_connection will not start.

At this stage, add the file to the folder with tests pytest.iniwhich will configure the log and enable hardpy when running pytest via plugin registration addopts = –hardpy-pt.

[pytest]
log_cli = true
log_cli_level = INFO
log_cli_format = %(asctime)s [%(levelname)s] %(message)s
log_cli_date_format = %H:%M:%S
addopts = --hardpy-pt

Launching the database

The test results must be saved. To do this HardPy uses a database CouchDB. It is important that the CouchDB version is not lower than 3.2.

Create a folder database and add the file there couchdb.inistoring the database settings.

[chttpd]
enable_cors=true

[cors]
origins = *
methods = GET, PUT, POST, HEAD, DELETE
credentials = true
headers = accept, authorization, content-type, origin, referer, x-csrf-token

To start, we will use the tool docker compose. Let's create a file docker-compose.yaml with the following contents:

version: "3.8"

services:
  couchserver:
    image: couchdb:3.3.2
    ports:
      - "5984:5984"
    environment:
      COUCHDB_USER: dev
      COUCHDB_PASSWORD: dev
    volumes:
      - ./dbdata:/opt/couchdb/data
      - ./couchdb.ini:/opt/couchdb/etc/local.ini

And we will launch the database through the command docker compose up.

The operator panel became available at http://127.0.0.1:5984/_utils with the login and password that were specified in docker-compose.yaml – dev/dev.

Launch the operator panel

The tests are written, the database is running, you can launch the operator panel, for this you need to run the command:

hardpy-panel .

or by command:

hardpy-panel <путь до папки tests>

if the launch does not occur from the tests folder.

As a result, in the terminal we should see a message that all 3 tests have been collected.

configfile: pytest.ini
plugins: hardpy-0.4.0, anyio-4.4.0
collected 3 items                                                                                                                        

<Dir tests>
  <Module test_1_fw.py>
    <Function test_fw_exist>
    <Function test_check_connection>
    <Function test_fw_flash>

======================================================= 3 tests collected in 0.37s =======================================================

And then you can open the operator panel in the browser at the address http://localhost:8000/

The Start button starts the tests, the results are in the database. In general, this is already a full-fledged test station for production, only the test coverage is still weak.

We will increase the volume of tests

Let's add 2 modules test_2_base_functions.py And test_3_user_button.pyWe will not analyze the content of the tests, except for the moments of using hardpy.

Checking the serial number and the presence of a jumper

• We read the serial number and check it.

• We check the presence of a jumper on the DUT.

# test_2_base_functions.py

import pytest
import hardpy

from dut_driver import DutDriver

pytestmark = pytest.mark.module_name("Base functions")

pytestmark = [
    pytest.mark.module_name("Base functions"),
    pytest.mark.dependency("test_1_fw::test_fw_flash"),
]

@pytest.mark.case_name("DUT info")
def test_serial_num(device_under_test: DutDriver):
    serial_number = device_under_test.reqserial_num()
    hardpy.set_dut_serial_number(serial_number)
    assert serial_number == "test_dut_1"

    info = {
        "name": serial_number,
        "batch": "batch_1",
    }
    hardpy.set_dut_info(info)

@pytest.mark.case_name("LED")
def test_jumper_closed(device_under_test: DutDriver):
    assert device_under_test.req_jumper_status() == 0

There are 2 functions used:

• set_dut_serial_number – records the DUT serial number into the database.

• set_dut_info – writes a dictionary with any information about the device to the database. In our case, we write the serial number and batch number again.

Check User button

We ask the user to click on User buttonand then enter the number of clicks.

# test_3_user_button.py

import pytest
import hardpy
from hardpy.pytest_hardpy.utils.dialog_box import (
    DialogBoxWidget,
    DialogBoxWidgetType,
    DialogBox,
)

from dut_driver import DutDriver

pytestmark = [
    pytest.mark.module_name("User interface"),
    pytest.mark.dependency("test_1_fw::test_fw_flash"),
]

@pytest.mark.case_name("User button")
def test_user_button(device_under_test: DutDriver):
    hardpy.set_message(f"Push the user button")
    keystroke = device_under_test.req_button_press()
    dbx = DialogBox(
        dialog_text=(
            f"Enter the number of times the button has "
            f"been pressed and press the Confirm button"
        ),
        title_bar="User button",
        widget=DialogBoxWidget(DialogBoxWidgetType.NUMERIC_INPUT),
    )
    user_input = int(hardpy.run_dialog_box(dbx))
    assert keystroke == user_input, (
        f"The DUT counted {keystroke} keystrokes"
        f"and the user entered {user_input} keystrokes"

The ability to call dialog boxes is used:

• Class DialogBox – describes the contents of the dialog box. In our case, this is a window for entering numerical values.

• run_dialog_box – calls up a dialog box on the operator panel side.

  

Collecting data

Writing reports to the database

To record final reports, you need to supplement the file conftest.py actions after testing is completed.
During testing, the report is always written to CouchDB, the database runstorebut we will add saving of all reports after testing to the database report.
The report is stored in the json document format according to the scheme described in documentation.

The database itself is available at http://127.0.0.1:5984/_utils/#

# conftest.py

import pytest
from hardpy import (
    CouchdbLoader,
    CouchdbConfig,
    get_current_report,
)

from dut_driver import DutDriver

@pytest.fixture(scope="session")
def device_under_test():
    dut = DutDriver("/dev/ttyACM0", 115200)
    yield dut

def finish_executing():
    report = get_current_report()
    if report:
        loader = CouchdbLoader(CouchdbConfig())
        loader.load(report)

@pytest.fixture(scope="session", autouse=True)
def fill_actions_after_test(post_run_functions: list):
    post_run_functions.append(finish_executing)
    yield

The following have been added to the file:

• Function fill_actions_after_testwhich fills the list post_run_functions functions that need to be executed after testing is complete.

• Function finish_executing – describes the steps for reading the current report from runstore and recording it in the database report.

  

Operator panel

How the stand operator sees it all:

Conclusion

Today we made a test station easily and simply, writing only a minimum of code, which is specific to DUT. The stand is ready for production.

Thanks to co-author @ilya_alexandrov!

P.S.
Where is the online collection and analysis of testing data and remote management of the test park? Everything will be soon, write if you want to participate in testing early versions.