What Should Be in Every FirmWare Repository
In this text, I propose to speculate what should be in a normal adult firmware repository (turnip / common fund) regardless of a specific project. That is, the most versatile and portable software components (bricks) that can be useful in almost any assembly.
Loader
The bootloader is needed to update the firmware without specialized equipment such as programmers. The bootloader must be able to update via UART. The remaining interfaces are updated according to the circumstances.
Logging control component
There must be a software component to manage logging to the UART. Coloring of logs in TeraTerm/Putty, selection/deselection of TimeStamp selection de-logging for a specific component. This will help to analyze the boot log of the device and follow the events inside the firmware at run-time (e), just by looking at the UART.
CLI (Command Line Interface) or Shell
CLI (shell) is a mandatory component for debugging and testing firmware. With this component, you can communicate with the device in human language. Many successful products have CLI (Flipper-Zero, NanoVNA, UbloxODIN).
Universal FIFO
In any Firmware project, sooner or later you will need a FIFO. For example, FIFO is needed to pre-storage data coming from UART Rx. FIFO is needed for the same CLI. FIFO implementation must adapt to any data type: char, uint16_t, array[N].
CRC checksum calculators
When implementing protocols, you will need a whole bunch of various checksums (CRC8, CRC16, CRC24, CRC32). CRCs are also needed for non-volatile file systems.
BSP (Board Support Package) for each family of microcontrollers
The codebase should be universal, but each microcontroller family will have its own unique BSP implementation. In BSP it is necessary to register implementations of the universal API. For example, to control GPIO, write OnChip FLASH, start timers, pull packets to I2C / SPI / I2S / UART, read ADC and more.
circular array
A cyclic array will be needed to implement digital filters, non-volatile logs.
Chip drivers
Each board has a lot of smart fancy chips with I2C/SPI/MDIO configuration. The repository should have a folder with a driver for each such chip and a separate folder with unit tests for the chip. Write in the comments the drivers of which smart SPI / I2C / MDIO chips you wrote.
Component to support each specific processor core
For each specific processor core (Cortex-M3/Cortex-M33/PowerPC/Tensilica Xtensa and pro) there should be a folder with description sorts for this core. These are the functions of on / off interrupts, reboot, printing the interrupt table, managing the system timer, and more.
Component to support each specific microcontroller
For each specific MCU, there should be a folder describing that MCU. This is primarily a list of pins, a description of the core, memory and available peripherals.
Component to support each specific board (platform code)
A separate folder in the repository must be created for each board. There should be configuration sources that say how many buttons, LEDs, SPI, I2C, SDIO are in this particular board. What kind of chip drivers are there and so on.
Separate folder for each specific assembly
All assemblies must share general code base. Therefore, in principle, there should not be *.c files in the project folder. Maximum one Makefile, a couple of configs in *.h, and a couple more scripts.
Software Timer
A software timer is a way to make hundreds of software timers from one hardware timer. Very useful if all the hardware timers are exhausted or their setting is very difficult. A good software timer allows you to set the counting period and phase and is fully configurable at run-time from the CLI.
Parsing strings
The CLI implementation needs functions to parse numbers of all data types from ASCII text strings. Therefore, data type parsers should be implemented for each data type. A kind of number interpreter. This is a very large part of the code.
Compression-decompression component
It may happen that the transceiver’s bit rate is very low (LoRa), and a lot of data needs to be transmitted. In this case, compression or data compression (for example, the LC3 codec) and decompression on the receiver side can save. The turnip must have a data compression codec software component.
Button debounce driver
Often in electronic boards there are clock buttons and reed switches. They create a rattle when closed. Therefore, the repository must have a button driver that suppresses contact bounce, can distinguish between a short press and a long, double one. All this will come in handy when debugging.
Software Schmitt Trigger
This link is for the implementation of hysteresis. Very useful for suppressing analog noise from sensors when relaying something.
limiter component
This is a function that ensures that a particular function is not called more often than set in the parameters. Very useful when implementing RTOS primitives. It is possible to skip all calls through the limiter directly in the superloop and thus set the response period for each function.
Implementation of the memory allocation and deallocation mechanism
It is very likely that you will have to dig your own reliable deterministic and portable version of the malloc and free functions, with the possibility of advanced diagnostics and garbage collection.
Component with math
A bit of linear algebra and numerical methods. For example, in automatic control systems on the MCU, a function is very necessary to calculate the angle between vectors (taking into account the sign, of course). And this immediately pulls up the implementation of scalar and vector multiplication. When working with DAC/DAC(s), it is useful to calculate the sine sample, PWM sample, Chirp(a), Saw(a), Fence(a). You may also need integer exponentiation and square root calculation. Sometimes you need to convert a 7-bit signed number or a 13-bit signed number to a standard int32_t. This also requires its own mathematical algorithms.
What mathematics did you have to implement in microcontrollers?
Software implementation of the calendar
If the project provides for a real-time clock, the board has a quartz and a battery, and the hardware clock inside the MCU can only increase the counter every second, then you will have to find and test a reliable software calendar. You give the number of seconds, you get the date and time, and vice versa.
Digital filters
Digital filters (DF) are needed not only for processing the audio stream and radar data. The digital filter will be needed to implement contact bounce suppression, to calculate the directions of movement of RFID beacons, and other things.
Cryptographer
Sooner or later, the firmware transmitted by the bootloader will have to be encrypted or stored in memory in encrypted form. Therefore, you should choose a trusted and tested data encryption algorithm, such as AES. In IT chips, AES is completely implemented in hardware and this component can be attributed to BSP.
Protocol parsers
The device will interact with the outside worlds. The same firmware must be altered according to some protocol (ModBus, yModem). There must be a software implementation of some protocol or series of protocols. I am more than sure that your company has some kind of proprietary binary protocol.
Non-volatile magazine
In any large C program, a lot of parameters and constants will accumulate, which will have to be varied, adjusted, calibrated. In order not to rebuild and reflash the gadget every time, you need to implement a non-volatile journal directly on the Target (e). Here’s War Story for you. The device hangs from the ceiling. Connected via LoRa to CLI. I registered a new parameter via CLI, reset (zero) the firmware via CLI and you have a new firmware. Easy.
Code generators
In programming microcontrollers, there is often a lot of repetitive code in structure. For example, parsing CAN matrices (8 bytes of payload data in a packet). Therefore, code generator utilities can be added to the repository to generate C-functions that parse packages with a known structure.
Unit Tests
All work will go to waste if the error in the next commit quietly goes unnoticed. A penny is worth a repository without tests. Therefore, it is necessary to cover the code with unit tests. Every non-trivial function should be covered by tests. Hardware-independent code can generally be tested directly on a PC.
Building from Make
Make is the most flexible way to manage cyclopean repositories. You can add / exclude thousands of files for thousands of assemblies with one line. Therefore, for each assembly, you must write a Makefile for each component *.mk file. Building from Make encourages modularity and component isolation.
You can add a bunch of dependency checks and assert(s) at the stage of bash scripts directly in *.mk files, since the make programming language supports conditional statements and functions. You can catch a lot of errors at the stage of working out the make utility. Stu Feldman (author of make) is a genius.
Autobuild scripts
Each assembly must be built from scripts. Scripts should write to OnChipFlash the hash of the latest commit and the name of the branch. This will then allow you to roll back and identify the cause of the bug.
I literally opened the folder with the project, clicked on the *.bat file, and after a maximum of 2 minutes I received artifacts *.hex *.bin *.elf *.map. Easy! Also, building from scripts will allow you to add builds to the Jenkins build server and monitor what is building and what is not every morning.
Auto firmware scripts
It should be possible to automatically flash Target. I clicked the flash.bat script from the project folder and fed the firmware to the microcontroller. And also the log was saved in a text file.
Cleanup and auto-formatting scripts
This is for combing the code with the clang-format utility. To keep the indentation predictable throughout the project. This will allow you to write simpler regular expressions for navigating through the code with the grep utility.
The auto-cleanup script at the root of the repository will remove temporary files with extensions *.d *.o *.obj *.bak *.i *.pp and free up a ton of disk space.
Auto generation of documentation
It is necessary to treat as programming not only the creation of artifacts. One must treat both programming also to create documentation. When developing Firmware, the documentation is a toolchain diagram, a block diagram of boards, a block diagram of complexes, and instructions. All these *.pdf, *.svg can be synthesized from dot code. Therefore, there must be makefile(s) to build the documentation. Text documents can be auto-generated in LaTeX
Conclusion
Summarizing the above, you can divide the code into hardware-dependent, hardware-independent and documentation. Plus some infrastructural assemblies for code generators and unit tests on dosktop.
Use CLI, version control systems, auto build systems and unit tests and everything will be fine and predictable for you.
If you think that there should be something else universal and useful in the Firmware repository, then write it in the comments.
I would also be grateful if you note in the comments which exotic microcontrollers and peripheral chips you personally programmed. Which repository management technologies do you consider promising, and which ones are on your black list.
Let’s build good repositories.
