Automatic Generation of Configurations for Make Assemblies

Prologue

When developing software (especially for microcontrollers), sooner or later you will have to face the need to somehow transfer configurations for a given software project.

In my experience, I have come to the conclusion that in terms of scaling the codebase, the easiest way to pass configs is through environment variables. Yes.. The advantage is that environment variables can be defined directly in scripts (Make, CMake, etc.).

It looks like this. Each assembly has a file config.mk which lists the software components from which this particular assembly should be assembled. The contents of this file usually look like this.


ADC=Y
ADC_ISR=Y
ADT=Y
ALLOCATOR=Y
ARRAY=Y
ASICS=Y

....

TASK=Y
TBFP=Y
TERMINAL=Y
TIME=Y
UART0=Y
UART2=Y
UNIT_TEST=Y
UTILS=Y
UWB=Y

These are just atomic lines. This is where the environment variables are defined. Declaratively lists what the firmware should be assembled from. The other assembly has its own declarative config.mk file and its own set of environment variables.

It is these same environment variables that decide which files to add to the compilation and which to exclude. For example, the environment variable SD_CARD=Y will add this make script to the assembly


ifneq ($(SD_CARD_MK_INC),Y)
    SD_CARD_MK_INC=Y

    $(info + SD card SPI driver)

    SD_CARD_DIR = $(ASICS_DIR)/sd_card
    #@echo $(error SD_CARD_DIR= $(SD_CARD_DIR))

    INCDIR += -I$(SD_CARD_DIR)

    OPT += -DHAS_CRC7
    OPT += -DHAS_CRC16
    OPT += -DHAS_SD_CARD
    OPT += -DHAS_SD_CARD_CRC7
    OPT += -DHAS_SD_CARD_CRC16

    SOURCES_C += $(SD_CARD_DIR)/sd_card_drv.c
    SOURCES_C += $(SD_CARD_DIR)/sd_card_crc.c
    SOURCES_C += $(SD_CARD_DIR)/sd_card_crc16.c

    ifeq ($(DIAG),Y)
        ifeq ($(SD_CARD_DIAG),Y)
            $(info + SD card diag)
            OPT += -DHAS_SD_CARD_DIAG
            SOURCES_C += $(SD_CARD_DIR)/sd_card_diag.c
        endif
    endif

    ifeq ($(CLI),Y)
        ifeq ($(SD_CARD_COMMANDS),Y)
            $(info + SD card commands)
            OPT += -DHAS_SD_CARD_COMMANDS
            SOURCES_C += $(SD_CARD_DIR)/sd_card_commands.c
        endif
    endif
endif

Enough inside config.mk register SD_CARD=Y and the scripts themselves will add everything you need to build C code: source code, paths to source code and preprocessor directives. You replaced one character in the scripts from “Y” to “N” and the make scripts automatically excluded the source code, source paths, and preprocessor directives for that particular software component from the build. Easy!

Environment variables inside config.mk, please note: sorted! And it’s very convenient to compare the configs of two different assemblies in the WinMerge utility.

As you know, computer programs are built hierarchically. For example, the software component of the CLI command line interface requires that the program already implement such software components as UART, TIMER, a number recognizer from a string, and other MISC little things.

What is the problem?
The problem is that there are a lot of environment variables (configs). If you build scripts using make, then you will have many environment variables for each specific build. Each config.mk will have 100 …. 200 lines and you can accidentally forget to set some important environment variable that activates some config. And without which the firmware will not work correctly at run-time. Here is an example of a typical config.mk

ADC=N
ADC_ISR=Y
ALLOCATOR=Y
ARRAY=Y
ASICS=Y
AUDIO=Y
BIN_2_STR=Y
BOARD=Y
BOARD_INFO=Y
BOARD_UTILS=Y
BUTTON=Y
CLI=Y
CLOCK=Y
CMSIS=Y
COMMON=Y
COMPLEX=Y
COMPONENTS=Y
CONNECTIVITY=Y
CORE=Y
CORE_APP=Y
CORE_EXT=Y
CORTEX_M33=Y
CRC16=Y
CRC8=Y
CRC=Y
CSV=Y
CUSTOM_PRINTF=Y
DATA_POC=Y
DEBUG=Y
DEBUGGER=Y
DFT=Y
DIAG=Y
DRIVERS=Y
DSP=Y
DYNAMIC_SAMPLES=Y
FIFO=Y
FIFO_CHAR=Y
FIFO_INDEX=Y
FLASH=Y
FLASH_EX=Y
FLASH_FS=Y
FLASH_FS_WRITE=Y
FLASH_WRITE=Y
GENERIC=Y
GPIO=Y
HEALTH_MONITOR=Y
I2C1=Y
I2C=Y
I2S0=Y
I2S0_MASTER=Y
I2S=Y
I2S_ISR=Y
INDICATION=Y
INTERFACES=Y
LED=Y
LED_MONO=Y
LED_VERIFY=Y
LIMITER=Y
LOG=Y
LOG_COLOR=Y
LOG_DIAG=Y
LOG_TIME_STAMP=Y
LOG_UTILS=Y
MATH=Y
MATH_VECTOR=Y
MCAL=Y
MCAL_NRF5340=Y
MICROCONTROLLER=Y
MISCELLANEOUS=Y
MULTIMEDIA=Y
NORTOS=Y
NRF5340=Y
NRF5340_APP=Y
NRF5340_DK=Y
NRFX=Y
NVIC_COMMANDS=Y
NVS=Y
NVS_WRITE=Y
PARAM=Y
PARAM_SET=Y
PCM_16_BIT=Y
PINS=Y
PROTOCOLS=Y
REAL_SAMPLE_ARRAY=N
SENSITIVITY=Y
SOFTWARE_TIMER=Y
STORAGE=Y
STR2_DOUBLE=Y
STREAM=Y
STRING=Y
STRING_PARSER=Y
SUPER_CYCLE=Y
SW_DAC=Y
SW_DAC_STATIC_SAMPLES=Y
SYSTEM=Y
SYSTICK=Y
SYS_INIT=Y
TABLE_UTILS=Y
TASK=Y
TERMINAL=Y
TEST_HW=Y
TEST_SW=Y
THIRD_PARTY=Y
TIME=Y
TIMER0=Y
TIMER1=Y
TIMER2=Y
TIMER=Y
UART0=Y
UART2=Y
UART=Y
UART_INTERRUPT=Y
UART_ISR=Y
UNIT_TEST=Y
WM8731=Y
WM8731_I2S_SLAVE=Y
WM8731_USB_MODE=Y
WM8731_VERIFY=Y
WRITE_ADDR=Y

In Zephyr Project, the problem of forgotten configs is partially solved by such a mechanism as KConfig. If you did not register the config, then KConfig will generate a build error or will automatically silently add the required config and continue building. However, unfortunately, there is no stand alone utility KConfig.exe that could be used in any assembly on Windows without relation to the Zephyr Project. Like this…

Solution

Obviously, we need to make sure that at the stage of working out make scripts, forgotten configs for the dependencies of those software components that we initially selected in the file are somehow automatically magically written config.mk.

It is necessary to make sure that the configs are registered automatically.

First of all, you need to create a separate make file for each software component, which will contain information about its dependencies. Otherwise, how will the assembly system figure out what else needs to be connected? You can name this file xxx_preconfig.mk. Here, for example, is the nvram_preconfig.mk file. Obviously, for the on-chip NVRAM code to work, software components such as CRC8 and MCAL for Flash peripherals are required. In this regard, the necessary environment variables are determined.

ifneq ($(NVRAM_PRECONFIG_INC),Y)
    NVRAM_PRECONFIG_INC=Y

    NVRAM=Y
    FLASH=Y
    NVS=Y
    NVRAM_PROC=Y
    CRC=Y
    CRC8=Y
endif

If the nvram_preconfig.mk script works in the build scripts, then you will not have to write CRC8=Y, FLASH=Y, NVS=Y, etc. in the config.mk file. All you have to do is write NVRAM=Y. Then everything else will appear on its own.

And here is the preconfig for the SD card in SPI mode.


ifneq ($(SD_CARD_PRECONFIG_MK_INC),Y)
    SD_CARD_PRECONFIG_MK_INC=Y

    CRC7=Y
    SPI=Y
    GPIO=Y
    CRC16=Y
    SD_CARD=Y
    SD_CARD_CRC7=Y
    SD_CARD_CRC16=Y
endif

In fact, the main idea of ​​this trick is taken from the CMake ideology. CMake collects the config, then the build system (make, ninja or IDE) builds the project itself. Only in this case everything is simpler. Both the config and the project are assembled by the build system itself! Here this is done by the omnivorous make utility.

Here is a simplified root file code_base.mk assembling the reused repository

ifneq ($(CODE_BASE_MK),Y)
    CODE_BASE_MK=Y

    include $(WORKSPACE_LOC)/code_base_preconfig.mk

    #preconfig/presets done!

    ifeq ($(CORE),Y)
        include $(WORKSPACE_LOC)/core/core.mk
    endif

    ifeq ($(MICROCONTROLLER),Y)
        include $(WORKSPACE_LOC)/microcontroller/microcontroller.mk
    endif

    ifeq ($(BOARD),Y)
        include $(WORKSPACE_LOC)/boards/boards.mk
    endif

    ifeq ($(THIRD_PARTY),Y)
        include $(WORKSPACE_LOC)/third_party/third_party.mk
    endif

    ifeq ($(APPLICATIONS),Y)
        include $(WORKSPACE_LOC)/applications/applications.mk
    endif

    ifeq ($(MCAL),Y)
        include $(WORKSPACE_LOC)/mcal/mcal.mk
    endif

    ifeq ($(ADT),Y)
        include $(WORKSPACE_LOC)/adt/adt.mk
    endif

    ifeq ($(CONNECTIVITY),Y)
        include $(WORKSPACE_LOC)/connectivity/connectivity.mk
    endif

    ifeq ($(CONTROL),Y)
        include $(WORKSPACE_LOC)/control/control.mk
    endif
    
    ifeq ($(COMPONENTS),Y)
        include $(WORKSPACE_LOC)/components/components.mk
    endif

    ifeq ($(COMPUTING),Y)
        include $(WORKSPACE_LOC)/computing/computing.mk
    endif

    ifeq ($(SENSITIVITY),Y)
        include $(WORKSPACE_LOC)/sensitivity/sensitivity.mk
    endif

    ifeq ($(STORAGE),Y)
        include $(WORKSPACE_LOC)/storage/storage.mk
    endif

    ifeq ($(SECURITY),Y)
        include $(WORKSPACE_LOC)/security/security.mk
    endif

    ifeq ($(ASICS),Y)
        include $(WORKSPACE_LOC)/asics/asics.mk
    endif

    ifeq ($(UNIT_TEST),Y)  
        include $(WORKSPACE_LOC)/unit_tests/unit_test.mk
    endif

    ifeq ($(MISCELLANEOUS),Y)
        include $(WORKSPACE_LOC)/miscellaneous/miscellaneous.mk
    endif
endif

note that code_base_preconfig.mk

include $(WORKSPACE_LOC)/code_base_preconfig.mk

called before the project itself is built. What is code_base_preconfig.mk? This is just a script for automatically arranging configs that we forgot about when compiling config.mk.

ifneq ($(CODE_BASE_PRECONFIG_MK),Y)
    CODE_BASE_PRECONFIG_MK=Y

    ifeq ($(BOARD),Y)
        include $(WORKSPACE_LOC)/boards/boards_preconfig.mk
    endif

    ifeq ($(MICROCONTROLLER),Y)
        include $(WORKSPACE_LOC)/microcontroller/microcontroller_preconfig.mk
    endif

    ifeq ($(CORE),Y)
        include $(WORKSPACE_LOC)/core/core_preconfig.mk
    endif

    ifeq ($(MCAL),Y)
        include $(WORKSPACE_LOC)/mcal/mcal_preconfig.mk
    endif

    ifeq ($(ADT),Y)
        include $(WORKSPACE_LOC)/adt/adt_preconfig.mk
    endif

    ifeq ($(CONNECTIVITY),Y)
        include $(WORKSPACE_LOC)/connectivity/connectivity_preconfig.mk
    endif

    ifeq ($(CONTROL),Y)
        include $(WORKSPACE_LOC)/control/control_preconfig.mk
    endif
    
    ifeq ($(COMPONENTS),Y)
        include $(WORKSPACE_LOC)/components/components_preconfig.mk
    endif

    ifeq ($(COMPUTING),Y)
        include $(WORKSPACE_LOC)/computing/computing_preconfig.mk
    endif

    ifeq ($(SENSITIVITY),Y)
        include $(WORKSPACE_LOC)/sensitivity/sensitivity_preconfig.mk
    endif

    ifeq ($(STORAGE),Y)
        include $(WORKSPACE_LOC)/storage/storage_preconfig.mk
    endif

    ifeq ($(SECURITY),Y)
        include $(WORKSPACE_LOC)/security/security_preconfig.mk
    endif

    include $(WORKSPACE_LOC)/asics/asics_preconfig.mk
    
endif

The same scripts nvram_preconfig.mk and sd_card_preconfig.mk will be called somewhere inside storage_preconfig.mk. Etc.

Thus, your original config.mk can be simplified to look like this

CLI=Y
COMPLEX=Y
CSV=Y
DEBUG=Y
NVRAM=Y
DEBUGGER=Y
DFT=Y
DIAG=Y
DSP=Y
DYNAMIC_SAMPLES=Y
GENERIC=Y
I2C1=Y
I2S0_MASTER=Y
NRF5340_DK=Y
NORTOS=Y
TASK=Y
TIMER0=Y
TIMER1=Y
TIMER2=Y
UART0=Y
UART2=Y
UNIT_TEST=Y
WM8731_I2S_SLAVE=Y
WM8731_USB_MODE=Y

Everything else will be preconfigured automatically! The root config config.mk has become 10 times simpler! Success!

Bottom line
As you can see, assembly from scripts provides such bonuses as the ability to automatically arrange configurations!

A technology has been developed for simple automatic transferable registration of dependency configs of software components based on the dependencies specified in the xxx_preconfig.mk files

I hope this text will help other programmers in developing their programs.

Links

Why is it Important to Collect Code from Scripts https://habr.com/ru/articles/723054/

Sorting Configs for Make Assemblies https://habr.com/ru/articles/745244/

Setting up ToolChain(s) for Win10+GCC+C+Makefile+ARM Cortex-Mx+GDBhttps://habr.com/ru/articles/673522/

Building firmware for CC2652 from Makefilehttps://habr.com/ru/articles/726352/

Generating dependencies inside the program https://habr.com/ru/articles/765424/

ToolChain: Setting up firmware assembly for Artery microcontrollers from Makefile https://habr.com/ru/articles/792590/

Automatic Firmware Version Update https://habr.com/ru/articles/791768/

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *