Djordje Nedic

Linker Script Generation for Firmware Projects: A Primer

2026-02-08T00:00:00+00:00

If you've worked on firmware for any length of time, chances are you had to modify or write a linker script at some point. The core principles are pretty straightforward - define memory regions, place sections inside, respect alignment requirements, define a few symbols and you're good to go.

While this is enough initially, projects can grow unpredictably. At some point you might need to add support for multiple chips or different external memories. Maybe you end up needing support for different bootloaders? Perhaps you might want to load the code into RAM directly for debugging purposes? Before you know it, these variations pile up, and suddenly you’re juggling a bunch of nearly identical linker scripts you have to keep in sync, which becomes tedious and error-prone.

In this article, we'll talk about how this happens and dive into a practical example that shows how to keep linker scripts manageable as firmware projects grow.

Note: If it’s been a while since you last tinkered with linker scripts, check out this great blogpost</a> and the official GNU documentation</a> </blockquote>
[TOC]
</a> Why variants break linker scripts</h2>
Consider a common scenario:

You start developing a product using a single chip and a straightforward memory layout. At this stage, your linker script is simple, easy to maintain, and just does what it needs to do. </li>

Later, the original chip becomes hard to source, so you add support for a pin-compatible replacement with more flash and RAM. To accommodate both chips, you copy the original linker script, changing only the region sizes. </li>

Next, you decide to add optional bootloader support for devices that can update over the air. This introduces another layer of variation: each chip can be built with or without the bootloader, doubling the number of linker scripts. </li>

Finally, a new variant of your product includes an LCD and requires external memory for bitmaps. Now you’re managing eight linker script variants across different configurations! </li> </ul>
This illustrates why linker scripts can become hard to maintain: each change has to be duplicated across all variations with some falling out of sync over time, making it harder to tell if changes are intentional or not. To stay in control, we have to find a way to approach them like other software: factor out the parts that stay the same, and only configure the parts that change.
</a> Building the mental framework</h2>
Let’s take a step back and figure out what really changes and what usually stays the same.
No matter what we do, the linker needs a valid script at the end of the day. That means the overall structure, syntax, and required sections are mostly fixed: we'll always have a .text</code> section, a .data</code> section, and certain architecture-specific details like alignment rules or symbol definitions. We could use these to create a template, which we can then fill out with configurable parts, such as
memory properties</li> section region mappings</li>reservations for optional features</li> </ul> As an example, the memory region part of the linker script template could look like this: MEMORY { FLASH (rx) : ORIGIN = ${flash_start}, LENGTH = ${flash_size} RAM (rwx): ORIGIN = ${ram_start}, LENGTH = ${ram_size} } </code></pre> Then, we can store the changing values in the build system itself or in separate YAML/JSON files and somehow generate the final linker script by combining them with the template. The output will still be a normal .ld</code> file - the linker should never see the template or the configuration. </a> A minimal example project</h2> Let’s look at a concrete example to see how all this works in practice. We’ll start from a simple CMake firmware project</a> for an STM32 microcontroller. For the impatient, the end result lives on the linker_script_generation branch</a>. For demonstration purposes, we will set aside the STM32CubeMX-provided linker script and use the simplest linker script that can still build and run the project correctly: ENTRY(Reset_Handler) MEMORY { RAM (xrw) : ORIGIN = 0x20000000, LENGTH = 20K FLASH (rx) : ORIGIN = 0x8000000, LENGTH = 64K } SECTIONS { .isr_vector : ALIGN(4) { KEEP(*(.isr_vector)) } > FLASH .text : ALIGN(4) { *(.text) *(.text*) KEEP (*(.init)) KEEP (*(.fini)) } > FLASH .preinit_array : ALIGN(4) { PROVIDE_HIDDEN(__preinit_array_start = .); KEEP(*(.preinit_array*)) PROVIDE_HIDDEN(__preinit_array_end = .); } > FLASH .init_array : ALIGN(4) { PROVIDE_HIDDEN(__init_array_start = .); KEEP(*(SORT(.init_array.*))) KEEP(*(.init_array)) PROVIDE_HIDDEN(__init_array_end = .); } > FLASH .fini_array : ALIGN(4) { PROVIDE_HIDDEN(__fini_array_start = .); KEEP(*(SORT(.fini_array.*))) KEEP(*(.fini_array)) PROVIDE_HIDDEN(__fini_array_end = .); } > FLASH .rodata : ALIGN(4) { *(.rodata) *(.rodata*) } > FLASH _sidata = LOADADDR(.data); .data : ALIGN(4) { _sdata = .; *(.data) *(.data*) _edata = ALIGN(4); } > RAM AT> FLASH .bss : ALIGN(4) { _sbss = .; __bss_start__ = _sbss; *(.bss) *(.bss*) _ebss = ALIGN(4); __bss_end__ = _ebss; } > RAM .heap (NOLOAD) : ALIGN(8) { PROVIDE ( end = . ); PROVIDE ( _end = . ); . = . + 0x200; } > RAM .stack (NOLOAD) : ALIGN(8) { /* A minimum beyond which linking will fail */ . = . + 0x400; } > RAM _estack = ORIGIN(RAM) + LENGTH(RAM); } </code></pre> </a> Turning the linker script into a template</h2> We could generate linker scripts from templates in a couple of ways. For instance, we could use Python and Jinja templates, CMake’s built-in configure_file()</code> to replace placeholders or even go wild and use sed</code>. In our case however, we'll be using the C preprocessor! It lives inside your toolchain, can substitute placeholders and conditionally include or not whole sections of text. Our first step is to factor out constants like the stack and heap size so that we can configure them dynamically: /* --snip-- */ MEMORY { RAM (xrw) : ORIGIN = RAM_ORIGIN, LENGTH = RAM_LENGTH FLASH (rx) : ORIGIN = FLASH_ORIGIN, LENGTH = FLASH_LENGTH } SECTIONS { /* --snip-- */ .heap (NOLOAD) : ALIGN(8) { PROVIDE ( end = . ); PROVIDE ( _end = . ); . = . + HEAP_SIZE; } > RAM .stack (NOLOAD) : ALIGN(8) { . = . + MINIMUM_STACK_SIZE; } > RAM /* --snip-- */ } </code></pre> and rename the linker script to linker.ld.in</code> to make it clear that that it is indeed a template. Now we can run the preprocessor to verify that we haven't broken anything: arm-none-eabi-gcc \ -E -P -x c \ -DRAM_ORIGIN=0x20000000 \ -DRAM_LENGTH=20K \ -DFLASH_ORIGIN=0x08000000 \ -DFLASH_LENGTH=64K \ -DHEAP_SIZE=0x200 \ -DMINIMUM_STACK_SIZE=0x400 \ linker.ld.in > linker.ld </code></pre> Note: The -E</code> argument invokes the preprocessor only, despite us calling the compiler binary, whereas -x c</code> flag forces the file to be treated as a preprocessor input, and -P</code> suppresses line markers which the preprocessor normally emits. </blockquote> If all goes well, the generated linker script should match the one we started with. Now that we know that this works, we can integrate it into our build system. First we will move these values into CMake variables. set(RAM_ORIGIN 0x20000000) set(RAM_LENGTH 20480) set(FLASH_ORIGIN 0x08000000) set(FLASH_LENGTH 65536) set(HEAP_SIZE 0x200) set(MINIMUM_STACK_SIZE 0x400) </code></pre> Note: We are making them purely numeric in order to be able to do operations on them later. </blockquote> Then, we can use a CMake custom command</a>: add_custom_command( OUTPUT ${CMAKE_BINARY_DIR}/linker.ld COMMAND ${CMAKE_C_COMPILER} -E -P -x c -DRAM_ORIGIN=${RAM_ORIGIN} -DRAM_LENGTH=${RAM_LENGTH} -DFLASH_ORIGIN=${FLASH_ORIGIN} -DFLASH_LENGTH=${FLASH_LENGTH} -DHEAP_SIZE=${HEAP_SIZE} -DMINIMUM_STACK_SIZE=${MINIMUM_STACK_SIZE} ${CMAKE_SOURCE_DIR}/linker.ld.in > ${CMAKE_BINARY_DIR}/linker.ld DEPENDS ${CMAKE_SOURCE_DIR}/linker.ld.in COMMENT "Generating linker script" ) </code></pre> Note: The DEPENDS</code> here makes sure that if the template itself changes, CMake understands that the command needs to be run again. </blockquote> and make it run at build time by making the executable depend on the generated linker script: add_executable(${EXECUTABLE} # --snip-- ${CMAKE_BINARY_DIR}/linker.ld ) </code></pre> Lastly, we can change the linking options to use our generated linker script: add_link_options( # --snip-- -T${CMAKE_BINARY_DIR}/linker.ld # --snip-- ) </code></pre> With this, our project builds and we can finally get to the fun part - making the linker script configurable! </a> Adding options</h2> Let's start simple by making the stack and heap size configurable. After deleting the constants we previously added, we need simply define cached values when configuring the project: cmake -S . -B build -DHEAP_SIZE=0 -DMINIMUM_STACK_SIZE=0x800 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=arm-none-eabi-gcc.cmake </code></pre> We can now check the resultant linker.ld</code> file in our build folder: /* --snip-- */ .heap (NOLOAD) : ALIGN(8) { PROVIDE ( end = . ); PROVIDE ( _end = . ); . = . + 0; } > RAM .stack (NOLOAD) : ALIGN(8) { . = . + 0x800; } > RAM /* --snip-- */ </code></pre> Nice! This should be our starting point when debugging any linker generation issues in the future. Since it is good practice to introduce default values for user configuration in CMake, we can handle the case when variables aren't defined: if (NOT DEFINED HEAP_SIZE) set(HEAP_SIZE 0x200) endif() if (NOT DEFINED MINIMUM_STACK_SIZE) set(MINIMUM_STACK_SIZE 0x400) endif() </code></pre> Next, let’s tackle optional bootloader support. The STM32 chip we are using</a> has uniform 1KB pages, so the bootloader could be sized any multiple of that. Additionally, some of those sectors could also be used for EEPROM emulation or other purposes. For the most general solution, we can handle this by introducing a FIRMWARE_FLASH_OFFSET</code> variable: if (DEFINED FIRMWARE_FLASH_OFFSET) math(EXPR FLASH_ORIGIN "${FLASH_ORIGIN} + ${FIRMWARE_FLASH_OFFSET}") math(EXPR FLASH_LENGTH "${FLASH_LENGTH} - ${FIRMWARE_FLASH_OFFSET}") endif() </code></pre> Another way to verify that our changes worked is to look at the .map</code> file in our build folder: .isr_vector 0x08001000 0x10c *(.isr_vector) .isr_vector 0x08001000 0x10c CMakeFiles/most_commented_embedded_cmakelists.elf.dir/startup_stm32f103xb.S.obj 0x08001000 g_pfnVectors .text 0x0800110c 0xbd4 </code></pre> In this case we used an offset of 4 Kilobytes and that is clearly reflected in the output. In some situations, you might want to upload the entire firmware into RAM. What you get this way is much faster firmware uploads, don't add wear to the flash and in some cases avoid the flash application image format requirements when bringing up new chips. To get a RAM-only firmware image, we have to replace the output section to region placement everywhere FLASH</code> is used. For example, .data</code> normally resides in RAM but is initialized from FLASH: .data : ALIGN(4) { _sdata = .; *(.data) *(.data*) _edata = ALIGN(4); } > RAM AT > FLASH </code></pre> For a RAM-only build, we change it to: .data : ALIGN(4) { _sdata = .; *(.data) *(.data*) _edata = ALIGN(4); } > RAM </code></pre> Since we're using the C preprocessor, we can just #ifdef</code> the mapping: .data : ALIGN(4) { _sdata = .; *(.data) *(.data*) _edata = ALIGN(4); #ifndef RAM_ONLY_BUILD } > RAM AT > FLASH #else } > RAM #endif </code></pre> To get the define from the user to the linker script preprocessing, first we want to add a configuration option and conditionally define RAM_ONLY_BUILD</code>. option(RAM_ONLY_BUILD "If set, the entire firmware is placed in RAM") </code></pre> and then conditionally define it for the linker script preprocessing using the BOOL</code> generator expression: add_custom_command( OUTPUT ${CMAKE_BINARY_DIR}/linker.ld COMMAND ${CMAKE_C_COMPILER} -E -P -x c # -- snip -- $<$<BOOL:${RAM_ONLY_BUILD}>:-DRAM_ONLY_BUILD> ${CMAKE_SOURCE_DIR}/linker.ld.in > ${CMAKE_BINARY_DIR}/linker.ld DEPENDS ${CMAKE_SOURCE_DIR}/linker.ld.in COMMENT "Generating the linker script" ) </code></pre> Note: If you haven't used generator expressions</a> previously, what this essentially means is: if the boolean representation of the RAM_ONLY_BUILD</code> value is true, add the text after >:</code>, otherwise not. </blockquote> Also, since non-zero initialized variables are normally copied from FLASH to RAM, we should also disable this behavior in the startup script: #ifndef RAM_ONLY_BUILD /* Copy the data segment initializers from flash to SRAM */ ldr r0, =_sdata ldr r1, =_edata ldr r2, =_sidata movs r3, #0 b LoopCopyDataInit CopyDataInit: ldr r4, [r2, r3] str r4, [r0, r3] adds r3, r3, #4 LoopCopyDataInit: adds r4, r0, r3 cmp r4, r1 bcc CopyDataInit #endif </code></pre> And pass the RAM_ONLY_BUILD</code> definition to the executable in CMake: target_compile_definitions(${EXECUTABLE} PRIVATE $<$<BOOL:${RAM_ONLY_BUILD}>:RAM_ONLY_BUILD> ) </code></pre> If you try building with our new option however, you might notice that the copy loop is still there: /* -- snip -- */ ldr r0, =_sdata 20000238: 480c ldr r0, [pc, #48] ; (2000026c <LoopFillZerobss+0x12>) ldr r1, =_edata 2000023a: 490d ldr r1, [pc, #52] ; (20000270 <LoopFillZerobss+0x16>) ldr r2, =_sidata 2000023c: 4a0d ldr r2, [pc, #52] ; (20000274 <LoopFillZerobss+0x1a>) movs r3, #0 2000023e: 2300 movs r3, #0 b LoopCopyDataInit 20000240: e002 b.n 20000248 <LoopCopyDataInit> 20000242 <CopyDataInit>: CopyDataInit: ldr r4, [r2, r3] 20000242: 58d4 ldr r4, [r2, r3] /* -- snip -- */ </code></pre> This is because by default, GCC does not preprocess assembly files with the lowercase .s extension, which the STM32CubeMX provided startup file uses. This goes away when we rename the file to use the .S</code> extension, intended for preprocessing. When we do this, we can see that the loop is gone: /* -- snip -- */ 20000274 <Reset_Handler>: cmp r4, r1 bcc CopyDataInit #endif /* Zero fill the bss segment. */ ldr r2, =_sbss 20000274: 4a07 ldr r2, [pc, #28] @ (20000294 <LoopFillZerobss+0x14>) ldr r4, =_ebss 20000276: 4c08 ldr r4, [pc, #32] @ (20000298 <LoopFillZerobss+0x18>) /* -- snip -- */ </code></pre> With this, we can place the entire firmware into RAM with a debugger! </a> Letting your imagination run wild</h2> At this point we've built a framework using a single, configurable linker script, with the build system being a single source of truth for both the linking and source-level configuration. From here, you can easily extend this to: Generate security or monitoring canaries between sections</li> Conditionally add logging or RTTI sections</li> Move things into tightly coupled memories conditionally</li> Enforce MPU or DMA required alignment based on where they are used</li> </ul> or go even further and split the single linker script template into multiple sources depending on the complexity of the project. </a> Real-world examples</h2> It might come as a no surprise that we're not the first ones to think of doing this - infact, several high-profile embedded projects already use linker script generation. For instance, the Zephyr RTOS</a> also generates its linker scripts from a collection of sources. The memory regions are defined by the device tree, while the sections are defined in a multitude of linker script fragments, which can come from the SoC architecture, the concrete SoC that is being used and even third party software modules! All of this is then processed by the C preprocessor with configuration variables originating from Zephyr's KConfig configuration system. </li> Another example would be the ESP-IDF SDK from Espressif, which has a deep and extensible</a> linker script generation process built with Python. Initial reasons for linker script generation in ESP-IDF come from external flash use - where one chip can be paired with differenly sized flash and the peculiarities of the full Harvard architecture of their Xtensa-based chips however it has grown to be an useful tool for customers too. It too uses KConfig and assembles the final linker script from fragments, some of which can come from user or third party code. </li> Finally, one (at least for me) unexpected project which uses linker script processing is the Linux kernel</a>! It has a C preprocessor pass on the linker scripts and uses KConfig variables for customization. </li> </ul> </a> Closing</h2> In this blogpost we've shown how regular linker scripts can be a limitation for complex firmware projects and built a simple framework for getting around them. Hopefully, this gives you a taste of what's possible and turns linker scripts from something you just have to deal with to something that is a regular part of the software you are building, or helps you debug an issue when working with some of the industry-standard projects which use linker script generation in the future!

Using nix-shell to create and share reproducible embedded development environments

2023-08-27T00:00:00+00:00

Historically, working on embedded software meant using proprietary manufacturer-provided IDEs with tools like the toolchain, build system and debugger all integrated. As open source tooling eventually caught up and even surpassed closed source solutions in many regards we often no longer have to rely on closed solutions and can adopt traditional software development tools and practices.
Unfortunately, this means that our development environment suddenly relies on a number of tools that have to work together and be correctly set up on every developer's system. Keeping track of this can be a time-consuming process and issues can arise, including implicit tool dependencies and incompatibilities from tool version mismatches.
In this post we will leverage the Nix</a> package manager to create a shareable and reproducible development environment, take a look at advantages it offers over existing solutions and by the end of the post, you should be able to quickly spin up a development environment, share it in the form of a single text file and be sure everyone using it will have the same exact setup.
[TOC]
</a> Our example project</h2>
In the previous blogpost</a>, we explored setting up a simple CMake project</a> for an STM32 microcontroller using open source tooling. To build the project, we need a few tools:

The ARM Embedded GCC toolchain</a> to compile and link the sources as well as bundle the standard library</li>
CMake</code> to generate our build system</li>
Make</code> or Ninja</code> to run the generated build system</li> </ul> Additionally, if we want to have a usable development environment, we need git</code> for version control and openocd</code> or another GDB server implementation for embedded in order to debug our firmware.</a> Challenges of manual tool management</h2> There are a couple of ways to manually obtain the necessary tools for building the project, let's go through some of them. </a> The system package manager</h3> One thing we could do is use our system's package repositories, for instance on Arch Linux we would do: sudo pacman -Syu cmake make git arm-none-eabi-gcc arm-none-eabi-binutils arm-none-eabi-newlib openocd </code></pre> This is quite convenient as it only involves a single command and has ensured compatibility with our system. There are however quite a few hidden downsides with this approach: Different developers working on the same project could be using different systems, leading to mismatches in tool versions</li> Developers could be updating their system at different times</li> The tools we need could be missing from our systems repositories</li> </ul> </a> Obtaining binary releases</h3> Alternatively, we could obtain binary releases of these tools. There is a bit more work involved with this approach, as we have to manually find, unpack and add all of these tools to our PATH</code>. This process can quickly become time consuming with the number of machines we have to perform the procedure on. We could forget to update one and have the same exact version mismatch issues. Additionally, we no longer have ensured compatibility with the systems the developers are using due to fixed dependencies on python versions, glibc versions or system folder structure expectations. </a> Third party repositories</h3> Finally, we can use third party repositories, like xPack</a>'s xpm</code>. This approach combines the benefits of the first two approaches, as we can still use a single command to install all of the tools and packagers try their best to ensure compatibility with all systems. We can observe however that some critical issues do not go away, like someone forgetting to update their packages. </a> Universal problems</h3> On top of issues specific to one of these approaches, the manual approach to tool management requires meticulous checking for compatibility between tools themselves and their various versions working together, which can be a very time consuming process. To sum everything up, our development environments are not self-contained, reproducible or managed. </a> Docker</h2> The most popular tool to deal with these issues at the moment is Docker</a>. Docker is a container runtime, created to run lightweight isolated environments in some ways akin to virtual machines on top of our system. What this means for us is that we can create images which will hold all of our tools and can be used to build and debug our project. </a> Building and using images</h3> Let's take a look at a minimal image recipe required to build our project in the form of a Dockerfile</code>: # Choose a base image FROM ubuntu:23.04 # Refresh the package index RUN apt-get -qq update # Get the required packages RUN apt-get -y install cmake make git gcc-arm-none-eabi binutils-arm-none-eabi libnewlib-arm-none-eabi # Specify the working directory inside the container WORKDIR /usr/project </code></pre> To build our image and give it a name, we can run: docker build -t most_commented_embedded_cmakelists . </code></pre> And to enter an interactive shell inside the container built from our image, we can use this command: docker run -v $(pwd):/usr/project -it most_commented_embedded_cmakelists </code></pre> Where the -v</code> argument maps our current working directory to the containers /usr/project</code> directory and -it</code> runs the container in the interactive mode. In order to be able to debug our project, we have to add openocd</code> and any debugger drivers to the list of packages: RUN apt-get -y install openocd libusb-1.0-0 </code></pre> then pass all usb devices through to the container every time we run it: docker run -v $(pwd):/usr/project -v /dev/bus/usb:/dev/bus/usb --privileged -it most_commented_embedded_cmakelists </code></pre> </a> Disadvantages of Docker</h3> Those observant enough have probably noticed a few downsides of this approach along the way. One major issue is that Docker images are not reproducible - the apt-get</code> command in the Dockerfile</code> gets the latest versions of the packages from the chosen base image repositories. This means that any time we run it, we may get different versions of packages. To combat this, we have to share images themselves, which can be rather large - the minimal image required to build our project alone is 2.9GB</code>. As an individual, we can upload these to Dockerhub</a> with an account, however businesses will have to pay a subscription for hosting these. Another issue is that we don't have access to our system's shell - our aliases, shell settings and more are not available to us in Docker images. Finally, it's easy to see that Docker commands can get very unwieldy due to the number of things we have to pass through to the container. </a> Enter Nix</h2> The Nix package manager released over 20 years ago but has recently caught the programming community spotlight with its ability to solve the dependency hell issues, work together with NixOS</a> to create reproducible Linux machines as well as create reproducible development environments. </a> nix-shell</h3> For this blogpost, we will be utilizing the nix-shell</a> command provided by the Nix package manager. Using nix-shell, you can: Create temporary environments for development, experimentation, or debugging without affecting your main system.</li> Share these environments with others in the form of a simple text file, ensuring everyone is on the same page.</li> </ol> A nix-shell</code> environment is usually defined with a shell.nix</code> or a default.nix</code> file containing a Nix language expression. Here's a basic example of what a shell.nix</code> file might look like for our embedded project: let pkgs = import <nixpkgs> {}; in pkgs.mkShell { packages = [ pkgs.gcc-arm-embedded pkgs.cmake pkgs.gnumake pkgs.git pkgs.openocd ]; } </code></pre> By running nix-shell</code> inside our project directory, the Nix package manager will download specified packages from the Nix repositories along with their dependencies and place them in /nix/store</code> and then export the executables of these packages to our shells PATH</code>. We can easily check this by running which</code> on one of our tools: which arm-none-eabi-gcc /nix/store/im3iiikm684j0dn166k78japxlknsrki-gcc-arm-embedded-12.2.rel1/bin/arm-none-eabi-gcc </code></pre> We can also run nix-shell</code> with the --pure</code> argument in order to verify that all of our tools are present and that building or debugging the project isn't using our system's tools by accident. This removes almost everything from our PATH before evaluating nix-shell</code>. Additionally, if we want to execute any commands in the shell on entry, we can use the shellHook</code> argument of mkShell</code>: pkgs.mkShell { # --snip-- shellHook = "echo \"Hello from our shell\""; } </code></pre> This can be useful if we want to start a server to talk to our device or automatically build the project upon entry. </a> Version pinning</h3> With this configuration, package versions can change every time we run nix-shell</code> depending on the package versions in the nixpkgs</code> collection however. To get true reproducibility, we have to somehow specify the package versions. Usually, this is done by pinning nixpkgs</code> to a specific point in time. This can be done in one of the following ways: By using builtins.fetchTarball</code> and specifying the nixpkgs</code> tarball</li> </ul> let pkgs = import (builtins.fetchTarball { url = "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz"; }) {}; </code></pre> By using builtins.fetchGit</code> and specifying a git revision</li> </ul> let pkgs = import (builtins.fetchGit { url = "https://github.com/nixos/nixpkgs/"; ref = "refs/heads/nixos-unstable"; rev = "976fa3369d722e76f37c77493d99829540d43845"; }) {}; </code></pre> While Nix itself does not provide an easy way to obtain these arguments based on the package versions we need the nix-versions</a> website can be used to obtain them and even generate code blocks shown above. Note: It is possible that mkShell</code> will fail when going too far back in time with nixpkgs</code> versions. This is because mkShell</code> was introduced in 2018 as a convenience wrapper around stdenv.mkDerivation</code> for nix-shell</code> purposes. Here is how we would achieve the same result using stdenv.mkDerivation</code>: pkgs.stdenv.mkDerivation { name = "my-shell"; buildInputs = [ pkgs.gcc-arm-embedded pkgs.cmake pkgs.gnumake pkgs.git pkgs.openocd ]; } </code></pre> </blockquote> If however the nixpkgs revision for one package does not provide the desired version of another package, it is possible to import multiple nixpkgs</code> revisions like so: let pkgs = import (builtins.fetchTarball { url = "https://github.com/NixOS/nixpkgs/archive/976fa3369d722e76f37c77493d99829540d43845.tar.gz"; }) {}; pkgs_arm_gcc = import (builtins.fetchTarball { url = "https://github.com/NixOS/nixpkgs/archive/b0f0b5c6c021ebafbd322899aa9a54b87d75a313.tar.gz"; }) {}; in pkgs.mkShell { packages = [ pkgs_arm_gcc.gcc-arm-embedded pkgs.cmake pkgs.gnumake pkgs.git pkgs.openocd ]; } </code></pre> With this, we have a fully reproducible and managed embedded development environment! This means that we can treat our development environment as code and anyone pulling our repository can instantly obtain the same environment we had when developing or deploying the project. </a> Using overlays</h3> Our example project so far required a relatively small number of tools compared to some manufacturer SDKs which can bundle a large number of tools and dependencies. An example of such an SDK would be ESP-IDF</a> from Espressif, which has a host of toolchains, python tools and dependencies. In situations like this, we may not want to manually specify our development environment, but instead rely on manufacturer or community maintained solutions and build on top of them. nixpkgs-esp-dev</a> offers such a solution for ESP-IDF, and looking at the code you will quickly realize the futility of maintaining such a solution yourself. To demonstrate the power of nixpkgs-esp-dev</code> we will use the esp-usb-bridge</a> project as an example. esp-usb-bridge</code> lets you create your own programmer and JTAG</code> probe with an ESP32-S2</code> or ESP32-S3</code> based board. After pulling the project, we can create a shell.nix</code> file according to the nixpkgs-esp-dev</code> instructions and pin the ESP-IDF version: let nixpkgs-esp-dev = builtins.fetchGit { url = "https://github.com/mirrexagon/nixpkgs-esp-dev.git"; rev = "4cc9ec3f8e992ed15924672192a2ce5fb0223121"; }; pkgs = import <nixpkgs> { overlays = [ (import "${nixpkgs-esp-dev}/overlay.nix") ]; }; in pkgs.mkShell { packages = [ pkgs.esp-idf-full ]; } </code></pre> From this point, we can build, flash and debug the project and continue expanding our new development environment with any new requirements we have by extending the shell.nix</code> file. Another such example is the rust-overlay</a> which lets you manage rust channels and architectures easily. These examples show us an important aspect of Nix - composability, we can import another .nix</code> file and use it as an expression. The overlay</code> argument lets us customize the nixpkgs</code> collection by overriding packages or introducing new ones. There are many community maintained overlays, which can also be combined to save you the effort of maintaining huge environments. </a> Third party tools</h3> Finally, there are quite a few projects built on top of nix-shell</code> that offer user convenience. lorri</a> for instance offers automatic nix-shell</code> invocation upon entering a project's directory providing seamless development, enables downloading newer package versions while in the shell itself and gives you tools to prevent the Nix garbage collector from sweeping away your nix-shell</code> obtained tools when invoked. </li> Another useful third party tool is devbox</a> which offers a convenient wrapper around nix-shell</code> and lets you forego the Nix language entirely, specifying your development environment in JSON. It also makes package pinning much more elegant and has a plugin system. </li> For those using VSCode, Nix Environment Selector</a> provides an elegant way to run your entire VSCode instance in the nix-shell</code> environment. </li> </ul> </a> Closing</h2> In this blogpost we covered some of the biggest challenges encountered by developers when switching to open tooling development environments, explored state of the art solutions and where they falter and then introduced nix-shell</code> as a viable alternative. At this point you should be ready to give nix-shell</code> a shot the next time you're required to manage an embedded development environment and save yourself and your team from the pains of manual tool management. For further reading you might be interested in reading about providing reproducible firmware builds</a> for your customers or partners, which Nix takes you a long way to or exploring Nix flakes</a> as a newer, experimental way to achieve the same goals.
The most thoroughly commented embedded CMakeLists file 2023-01-05T00:00:00+00:00 Many embedded software engineers have had to deal with build systems that are either non cross-platform or a part of the IDE they are using, don't offer easy composability or per-library configuration, provide no testing support or ways to generate or package files without using a third party scripting language. CMake</a> is a build system that offers solutions to many of those problems and is already well adopted in the world of traditional software development. Inspired by The most thoroughly commented linker script</a> blogpost and having used CMake in two different professional environments, I have decided to write an overview of a minimal CMakeLists file for an embedded project. Here</a> is the CMakeLists file we are going to be taking a look at. It is a part of a CMake STM32 project</a> i created as a demonstration and uses an STM32F103 MCU. </a> The minimum version</h2> cmake_minimum_required(VERSION 3.21) </code></pre> This call is required at the top of every CMakeLists file, it sets the minimum CMake version our project can be built with, and this decides the features that can be used. Generally it is advisable to set it as low as possible until features from the newer versions are required. Another good approach is to check repology</a> for the CMake versions shipped in most used Linux distributions. </a> Project declaration</h2> project(most_commented_embedded_cmakelists VERSION 1.0.0 DESCRIPTION "This is a demo project" ) </code></pre> The project()</a> call sets up the project name and optionally version, description, homepage and more and stores them in variables that can be used later. For instance, the project name is stored in PROJECT_NAME</code>, description in PROJECT_DESCRIPTION</code> and so on. </a> Language settings</h2> enable_language(C ASM) </code></pre> This sets the languages the project is going to be using, if C++ support is desired add CXX</code> to the list. set(CMAKE_C_STANDARD 11) set(CMAKE_C_STANDARD_REQUIRED ON) </code></pre> This sets the language standard globally and whether the standard is a requirement. This may not be required, however due to the non-backward compatible nature of C and C++ it is preferred. C11 is a good first choice for projects as it provides useful additions like atomics</a>, however it does away with optional features like VLAs</a>, so C99 or lower might be preferable when working with legacy projects. set(CMAKE_C_EXTENSIONS OFF) </code></pre> This controls the inclusion of the GNU extensions</a> to the C language globally, OFF</code> is preferred for more standard and portable C however GNU extensions can provide useful things on top of the base standard. </a> Compiler options</h2> set(MCU_OPTIONS -mcpu=cortex-m3 -mthumb ) </code></pre> This sets a custom microcontroller specific compiler flags</a> variable. First, the cpu variant is set and then the compiler is told to emit thumb instructions</a>. set(EXTRA_OPTIONS -fdata-sections -ffunction-sections ) </code></pre> This sets up a variable containing extra functionality compiler flags. The flags added here ensure that all objects are placed in separate linker sections. We will see later why that is important. set(OPTIMIZATION_OPTIONS $<$<CONFIG:Debug>:-Og> ) </code></pre> Here the compiler optimization flags custom variable is created. The weird looking expression conditionally sets compiler optimization level to -Og</code> when building with the Debug configuration</a>. As most microcontrollers are rather limited in terms of flash size and cpu speed, the default optimization level of -O0</code> is not suitable. -Og</code> is an optimization level created for the best tradeoff between size, speed and debugging viability. Other configurations use their default compiler optimization levels, -O2</code> for the Release</code> configuration and -Os</code> for the MinSizeRel</code> configuration, so nothing else is added. set(DEPENDENCY_INFO_OPTIONS </code></pre> Here the preprocessor flags custom variable containing dependency info options is created. -MMD </code></pre> This line tells the preprocessor to generate dependency files for Make-compatible build systems instead of full preprocessor output, while removing mentions to system header files. Note: If we need the preprocessor output ourselves we can pass the -E</code> argument to the compiler instead. </blockquote> -MP </code></pre> This option instructs the preprocessor to add a phony target for each dependency other than the main file to work around errors the build system gives if you remove header files without updating it. -MF "$(@:%.o=%.d)" ) </code></pre> The last line specifies that we want the dependency files to be generated with the same name as the corresponding object file. set(DEBUG_INFO_OPTIONS -g3 -gdwarf-2 ) </code></pre> This creates a custom variable containing compiler flags for generating debug information</a>. The first line tells the compiler to produce the debugging output and the level of detail while the second line configures the debug output format and version. We are using dwarf version 2 for best debugger compatibility. add_compile_options( ${MCU_OPTIONS} ${EXTRA_OPTIONS} ${DEBUG_INFO_OPTIONS} ${DEPENDENCY_INFO_OPTIONS} ${OPTIMIZATION_OPTIONS} ) </code></pre> Finally, all the created variables are used to set the global compiler options. Note: It is possible to override these per-target by using target_compile_options()</code>, for instance to apply a higher optimization level to third party libraries we are not going to be debugging. </blockquote> </a> Linker options</h2> add_link_options( </code></pre> Here we are adding global linker options. ${MCU_OPTIONS} </code></pre> First, it is required to pass the previously created microcontroller specific flags variable created earlier. -specs=nano.specs </code></pre> This tells the linker to include the nano variant of the newlib</a> standard library which is optimized for minimal binary size and RAM use. The regular newlib variant is used simply by not passing this. -T${CMAKE_SOURCE_DIR}/STM32F103C8Tx_FLASH.ld </code></pre> Here the linkerscript of the chip is passed. The linkerscript tells the linker where to store the objects in memory. I recommend reading the already mentioned most thoroughly commented linker script</a> blogpost. -Wl,-Map=${PROJECT_NAME}.map,--cref </code></pre> This directive instructs the linker to generate a mapfile. Mapfiles contain information about the final layout of the firmware binary and are an invaluable resource for development. I recommend reading this excellent Interrupt blogpost</a> for a quick introduction. -Wl,--gc-sections ) </code></pre> This is the linker flag for removing the unused sections from the final binary, it works in conjunction with the previously mentioned -fdata-sections</code> and -ffunction-sections</code> compiler flags to remove all unused objects from the final binary. </a> Linking the standard library</h2> link_libraries("-lc -lm -lnosys") </code></pre> This call tells the linker to link the standard library components to all the libraries and executables added after. The order of the standard library calls is important as libraries can only see symbols from libraries on their right: -lc is the libc containing most standard library features</li> -lm is the libm containing math functionality</li> -lnosys provides stubs for the syscalls, essentially placeholders for what would be operating system calls</li> </ul> Note: CMake also offers the CMAKE_C_STANDARD_LIBRARIES</code> variable, whose value is always appended at the end of the final linking command. </blockquote> </a> Adding library subprojects</h2> add_subdirectory(Drivers) </code></pre> When a subdirectory is added, the CMakeLists file contained in that subdirectory is evaluated, this is usually chained for composability. In this case the Drivers</code> CMakeLists file creates a static library called Drivers</code>. target_include_directories(Drivers PRIVATE Core/Inc ) </code></pre> We need to include the Core/Inc</code> for the Drivers</code> library, as it depends on the definitions inside the Core</code> headers. </a> The executable</h2> set(EXECUTABLE ${PROJECT_NAME}.elf) </code></pre> This creates a variable containing the executable name by appending the ELF</a> extension to the project name. add_executable(${EXECUTABLE} Core/Src/main.c Core/Src/stm32f1xx_hal_msp.c Core/Src/stm32f1xx_it.c Core/Src/system_stm32f1xx.c startup_stm32f103xb.s ) </code></pre> Here we are creating the executable target and adding top-level sources to it. An executable is the final product of the build process, in this case our firmware. Note the startup file gets added along with the C sources. target_include_directories(${EXECUTABLE} PRIVATE Core/Inc ) </code></pre> Add the include directories for the executable. Since we don't need the includes to propagate up as our executable is the final product of the build process, we are including as PRIVATE</code>. target_compile_options(${EXECUTABLE} PRIVATE -Wall -Wextra -Wshadow -Wconversion -Wdouble-promotion ) </code></pre> This adds additional warning compiler flags</a> to our executable target, enabling them for our sources, but not third party libraries. All of these provide much more safety if not ignored. target_link_libraries(${EXECUTABLE} PRIVATE Drivers ) </code></pre> Here we link the libraries to the executable. Same principle applies to the reasoning behind PRIVATE</code>. </a> Post-build commands</h2> add_custom_command(TARGET ${EXECUTABLE} POST_BUILD COMMAND ${CMAKE_SIZE_UTIL} ${EXECUTABLE} ) </code></pre> This creates a custom command that prints out the firmware binary size information. Example: text data bss dec hex filename 3432 20 1572 5024 13a0 most_commented_embedded_cmakelists.elf </code></pre> text</code> is the code, data</code> stores variables that have a non-zero initial value and have to be stored in flash, bss</code> stores zero initial values that only take up ram. dec</code> and hex</code> are just the cumulative size in decimal and hexadecimal notation respectively. Here</a> is a good resource for understanding these sections and to learn more about output options. add_custom_command(TARGET ${EXECUTABLE} POST_BUILD COMMAND ${CMAKE_OBJCOPY} -O ihex ${EXECUTABLE} ${PROJECT_NAME}.hex COMMAND ${CMAKE_OBJCOPY} -O binary ${EXECUTABLE} ${PROJECT_NAME}.bin ) </code></pre> This creates a custom command to generate binary and hex files. These can be used depending on which method of loading the firmware to the MCU is used. </a> Closing</h2> Hopefully this gives you an overview of how a minimal CMakeLists file for an embedded project looks like and peaked your curiosity if you've never used CMake for embedded projects. It is worth keeping in mind that this article is not a complete guide to CMake and doesn't go into toolchain files, library CMakeLists and other things required for a complete CMake-based embedded project, however the project on GitHub</a> used for demonstration contains all of those and can be used as a template.