4 This document explains the Espressif IoT Development Framework build system and the
5 concept of "components"
7 Read this document if you want to know how to organise a new ESP-IDF project.
9 We recommend using the esp-idf-template_ project as a starting point for your project.
11 Using the Build System
12 ======================
14 The esp-idf README file contains a description of how to use the build system to build your project.
19 An ESP-IDF project can be seen as an amalgamation of a number of components.
20 For example, for a webserver that shows the current humidity, there could be:
22 - The ESP32 base libraries (libc, rom bindings etc)
25 - The FreeRTOS operating system
27 - A driver for the humidity sensor
28 - Main code tying it all together
30 ESP-IDF makes these components explicit and configurable. To do that,
31 when a project is compiled, the build environment will look up all the
32 components in the ESP-IDF directories, the project directories and
33 (optionally) in additional custom component directories. It then
34 allows the user to configure the ESP-IDF project using a a text-based
35 menu system to customize each component. After the components in the
36 project are configured, the build process will compile the project.
41 - A "project" is a directory that contains all the files and configuration to build a single "app" (executable), as well as additional supporting output such as a partition table, data/filesystem partitions, and a bootloader.
43 - "Project configuration" is held in a single file called sdkconfig in the root directory of the project. This configuration file is modified via ``make menuconfig`` to customise the configuration of the project. A single project contains exactly one project configuration.
45 - An "app" is an executable which is built by esp-idf. A single project will usually build two apps - a "project app" (the main executable, ie your custom firmware) and a "bootloader app" (the initial bootloader program which launches the project app).
47 - "components" are modular pieces of standalone code which are compiled into static libraries (.a files) and linked into an app. Some are provided by esp-idf itself, others may be sourced from other places.
49 Some things are not part of the project:
51 - "ESP-IDF" is not part of the project. Instead it is standalone, and linked to the project via the ``IDF_PATH`` environment variable which holds the path of the ``esp-idf`` directory. This allows the IDF framework to be decoupled from your project.
53 - The toolchain for compilation is not part of the project. The toolchain should be installed in the system command line PATH, or the path to the toolchain can be set as part of the compiler prefix in the project configuration.
59 An example project directory tree might look like this::
64 - components/ - component1/ - component.mk
67 - component2/ - component.mk
70 - include/ - component2.h
77 This example "myProject" contains the following elements:
79 - A top-level project Makefile. This Makefile set the ``PROJECT_NAME`` variable and (optionally) defines
80 other project-wide make variables. It includes the core ``$(IDF_PATH)/make/project.mk`` makefile which
81 implements the rest of the ESP-IDF build system.
83 - "sdkconfig" project configuration file. This file is created/updated when "make menuconfig" runs, and holds configuration for all of the components in the project (including esp-idf itself). The "sdkconfig" file may or may not be added to the source control system of the project.
85 - Optional "components" directory contains components that are part of the project. A project does not have to contain custom components of this kind, but it can be useful for structuring reusable code or including third party components that aren't part of ESP-IDF.
87 - "main" directory is a special "pseudo-component" that contains source code for the project itself. "main" is a default name, the Makefile variable ``SRCDIRS`` defaults to this but can be set to look for pseudo-components in other directories.
89 - "build" directory is where build output is created. After the make process is run, this directory will contain interim object files and libraries as well as final binary output files. This directory is usually not added to source control or distributed with the project source code.
91 Component directories contain a component makefile - ``component.mk``. This may contain variable definitions
92 to control the build process of the component, and its integration into the overall project. See `Component Makefiles` for more details.
94 Each component may also include a ``Kconfig`` file defining the `component configuration` options that can be set via the project configuration. Some components may also include ``Kconfig.projbuild`` and ``Makefile.projbuild`` files, which are special files for `overriding parts of the project`.
99 Each project has a single Makefile that contains build settings for the entire project. By default, the project Makefile can be quite minimal.
101 Minimal Example Makefile
102 ^^^^^^^^^^^^^^^^^^^^^^^^
106 PROJECT_NAME := myProject
108 include $(IDF_PATH)/make/project.mk
111 Mandatory Project Variables
112 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
114 - ``PROJECT_NAME``: Name of the project. Binary output files will use this name - ie myProject.bin, myProject.elf.
116 Optional Project Variables
117 ^^^^^^^^^^^^^^^^^^^^^^^^^^
119 These variables all have default values that can be overridden for custom behaviour. Look in ``make/project.mk`` for all of the implementation details.
121 - ``PROJECT_PATH``: Top-level project directory. Defaults to the directory containing the Makefile. Many other project variables are based on this variable. The project path cannot contain spaces.
122 - ``BUILD_DIR_BASE``: The build directory for all objects/libraries/binaries. Defaults to ``$(PROJECT_PATH)/build``.
123 - ``COMPONENT_DIRS``: Directories to search for components. Defaults to `$(IDF_PATH)/components`, `$(PROJECT_PATH)/components` and ``EXTRA_COMPONENT_DIRS``. Override this variable if you don't want to search for components in the esp-idf & project ``components`` directories.
124 - ``EXTRA_COMPONENT_DIRS``: Optional list of additional directories to search for components. Components themselves are in sub-directories of these directories, this is a top-level directory containing the component directories.
125 - ``COMPONENTS``: A list of component names to build into the project. Defaults to all components found in the COMPONENT_DIRS directories.
126 - ``SRCDIRS``: Directories under the main project directory which contain project-specific "pseudo-components". Defaults to 'main'. The difference between specifying a directory here and specifying it under ``EXTRA_COMPONENT_DIRS`` is that a directory in ``SRCDIRS`` is a component itself (contains a file "component.mk"), whereas a directory in ``EXTRA_COMPONENT_DIRS`` contains component directories which contain a file "component.mk". See the `Example Project` for a concrete case of this.
132 Each project contains one or more components, which can either be part of esp-idf or added from other component directories.
134 A component is any sub-directory that contains a `component.mk` file.[#f1]_.
136 Minimal Component Makefile
137 ^^^^^^^^^^^^^^^^^^^^^^^^^^
139 The minimal ``component.mk`` file is an empty file(!). If the file is empty, the default component behaviour is set:
141 - All source files in the same directory as the makefile (``*.c``, ``*.cpp``, ``*.S``) will be compiled into the component library
142 - A sub-directory "include" will be added to the global include search path for all other components.
143 - The component library will be linked into the project app.
145 See `example component makefiles` for more complete component makefile examples.
147 Note that there is a different between an empty ``component.mk`` file (which invokes default component build behaviour) and no ``component.mk`` file (which means no default component build behaviour will occur.) It is possible for a component to have no `component.mk` file, if it only contains other files which influence the project configuration or build process.
149 .. component variables:
151 Preset Component Variables
152 ^^^^^^^^^^^^^^^^^^^^^^^^^^
154 The following component-specific variables are available for use inside ``component.mk``, but should not be modified:
156 - ``COMPONENT_PATH``: The component directory. Evaluates to the absolute path of the directory containing ``component.mk``. The component path cannot contain spaces.
157 - ``COMPONENT_NAME``: Name of the component. Defaults to the name of the component directory.
158 - ``COMPONENT_BUILD_DIR``: The component build directory. Evaluates to the absolute path of a directory inside `$(BUILD_DIR_BASE)` where this component's source files are to be built. This is also the Current Working Directory any time the component is being built, so relative paths in make targets, etc. will be relative to this directory.
159 - ``COMPONENT_LIBRARY``: Name of the static library file (relative to the component build directory) that will be built for this component. Defaults to ``$(COMPONENT_NAME).a``.
161 The following variables are set at the project level, but exported for use in the component build:
163 - ``PROJECT_NAME``: Name of the project, as set in project Makefile
164 - ``PROJECT_PATH``: Absolute path of the project directory containing the project Makefile.
165 - ``COMPONENTS``: Name of all components that are included in this build.
166 - ``CONFIG_*``: Each value in the project configuration has a corresponding variable available in make. All names begin with ``CONFIG_``.
167 - ``CC``, ``LD``, ``AR``, ``OBJCOPY``: Full paths to each tool from the gcc xtensa cross-toolchain.
168 - ``HOSTCC``, ``HOSTLD``, ``HOSTAR``: Full names of each tool from the host native toolchain.
170 If you modify any of these variables inside ``component.mk`` then this will not prevent other components from building but it may make your component hard to build and/or debug.
172 Optional Project-Wide Component Variables
173 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
175 The following variables can be set inside ``component.mk`` to control build settings across the entire project:
177 - ``COMPONENT_ADD_INCLUDEDIRS``: Paths, relative to the component
178 directory, which will be added to the include search path for
179 all components in the project. Defaults to ``include`` if not overridden. If an include directory is only needed to compile
180 this specific component, add it to ``COMPONENT_PRIV_INCLUDEDIRS`` instead.
181 - ``COMPONENT_ADD_LDFLAGS``: Add linker arguments to the LDFLAGS for
182 the app executable. Defaults to ``-l$(COMPONENT_NAME)``. If
183 adding pre-compiled libraries to this directory, add them as
184 absolute paths - ie $(COMPONENT_PATH)/libwhatever.a
185 - ``COMPONENT_DEPENDS``: Optional list of component names that should
186 be compiled before this component. This is not necessary for
187 link-time dependencies, because all component include directories
188 are available at all times. It is necessary if one component
189 generates an include file which you then want to include in another
190 component. Most components do not need to set this variable.
191 - ``COMPONENT_ADD_LINKER_DEPS``: Optional list of component-relative paths
192 to files which should trigger a re-link of the ELF file if they change.
193 Typically used for linker script files and binary libraries. Most components do
194 not need to set this variable.
196 The following variable only works for components that are part of esp-idf itself:
198 - ``COMPONENT_SUBMODULES``: Optional list of git submodule paths
199 (relative to COMPONENT_PATH) used by the component. These will be
200 checked (and initialised if necessary) by the build process. This
201 variable is ignored if the component is outside the IDF_PATH
205 Optional Component-Specific Variables
206 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
208 The following variables can be set inside ``component.mk`` to control the build of that component:
210 - ``COMPONENT_PRIV_INCLUDEDIRS``: Directory paths, must be relative to
211 the component directory, which will be added to the include search
212 path for this component's source files only.
213 - ``COMPONENT_EXTRA_INCLUDES``: Any extra include paths used when
214 compiling the component's source files. These will be prefixed with
215 '-I' and passed as-is to the compiler. Similar to the
216 ``COMPONENT_PRIV_INCLUDEDIRS`` variable, except these paths are not
217 expanded relative to the component directory.
218 - ``COMPONENT_SRCDIRS``: Directory paths, must be relative to the
219 component directory, which will be searched for source files (``*.cpp``,
220 ``*.c``, ``*.S``). Defaults to '.', ie the component directory
221 itself. Override this to specify a different list of directories
222 which contain source files.
223 - ``COMPONENT_OBJS``: Object files to compile. Default value is a .o
224 file for each source file that is found in ``COMPONENT_SRCDIRS``.
225 Overriding this list allows you to exclude source files in
226 ``COMPONENT_SRCDIRS`` that would otherwise be compiled. See
227 `Specifying source files`
228 - ``COMPONENT_EXTRA_CLEAN``: Paths, relative to the component build
229 directory, of any files that are generated using custom make rules
230 in the component.mk file and which need to be removed as part of
231 ``make clean``. See `Source Code Generation` for an example.
232 - ``COMPONENT_OWNBUILDTARGET`` & `COMPONENT_OWNCLEANTARGET`: These
233 targets allow you to fully override the default build behaviour for
234 the component. See `Fully Overriding The Component Makefile` for
236 - ``CFLAGS``: Flags passed to the C compiler. A default set of
237 ``CFLAGS`` is defined based on project settings. Component-specific
238 additions can be made via ``CFLAGS +=``. It is also possible
239 (although not recommended) to override this variable completely for
241 - ``CPPFLAGS``: Flags passed to the C preprocessor (used for .c, .cpp
242 and .S files). A default set of ``CPPFLAGS`` is defined based on
243 project settings. Component-specific additions can be made via
244 ``CPPFLAGS +=``. It is also possible (although not recommended) to
245 override this variable completely for a component.
246 - ``CXXFLAGS``: Flags passed to the C++ compiler. A default set of
247 ``CXXFLAGS`` is defined based on project
248 settings. Component-specific additions can be made via ``CXXFLAGS
249 +=``. It is also possible (although not recommended) to override
250 this variable completely for a component.
252 To apply compilation flags to a single source file, you can add a variable override as a target, ie::
254 apps/dhcpserver.o: CFLAGS += -Wno-unused-variable
256 This can be useful if there is upstream code that emits warnings.
258 Component Configuration
259 -----------------------
261 Each component can also have a Kconfig file, alongside ``component.mk``. This contains contains
262 configuration settings to add to the "make menuconfig" for this component.
264 These settings are found under the "Component Settings" menu when menuconfig is run.
266 To create a component KConfig file, it is easiest to start with one of the KConfig files distributed with esp-idf.
268 For an example, see `Adding conditional configuration`.
270 Build Process Internals
271 -----------------------
273 Top Level: Project Makefile
274 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
276 - "make" is always run from the project directory and the project makefile, typically named Makefile.
277 - The project makefile sets ``PROJECT_NAME`` and optionally customises other `optional project variables`
278 - The project makefile includes ``$(IDF_PATH)/make/project.mk`` which contains the project-level Make logic.
279 - ``project.mk`` fills in default project-level make variables and includes make variables from the project configuration. If the generated makefile containing project configuration is out of date, then it is regenerated (via targets in ``project_config.mk``) and then the make process restarts from the top.
280 - ``project.mk`` builds a list of components to build, based on the default component directories or a custom list of components set in `optional project variables`.
281 - Each component can set some `optional project-wide component variables`. These are included via generated makefiles named ``component_project_vars.mk`` - there is one per component. These generated makefiles are included into ``project.mk``. If any are missing or out of date, they are regenerated (via a recursive make call to the component makefile) and then the make process restarts from the top.
282 - `Makefile.projbuild` files from components are included into the make process, to add extra targets or configuration.
283 - By default, the project makefile also generates top-level build & clean targets for each component and sets up `app` and `clean` targets to invoke all of these sub-targets.
284 - In order to compile each component, a recursive make is performed for the component makefile.
286 To better understand the project make process, have a read through the ``project.mk`` file itself.
288 Second Level: Component Makefiles
289 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
291 - Each call to a component makefile goes via the ``$(IDF_PATH)/make/component_wrapper.mk`` wrapper makefile.
292 - The ``component_wrapper.mk`` is called with the current directory set to the component build directory, and the ``COMPONENT_MAKEFILE`` variable is set to the absolute path to ``component.mk``.
293 - ``component_wrapper.mk`` sets default values for all `component variables`, then includes the `component.mk` file which can override or modify these.
294 - If ``COMPONENT_OWNBUILDTARGET`` and ``COMPONENT_OWNCLEANTARGET`` are not defined, default build and clean targets are created for the component's source files and the prerequisite ``COMPONENT_LIBRARY`` static library file.
295 - The ``component_project_vars.mk`` file has its own target in ``component_wrapper.mk``, which is evaluated from ``project.mk`` if this file needs to be rebuilt due to changes in the component makefile or the project configuration.
297 To better understand the component make process, have a read through the ``component_wrapper.mk`` file and some of the ``component.mk`` files included with esp-idf.
299 Debugging The Make Process
300 --------------------------
302 Some tips for debugging the esp-idf build system:
304 - Appending ``V=1`` to the make arguments (or setting it as an environment variable) will cause make to echo all commands executed, and also each directory as it is entered for a sub-make.
305 - Running ``make -w`` will cause make to echo each directory as it is entered for a sub-make - same as ``V=1`` but without also echoing all commands.
306 - Running ``make --trace`` (possibly in addition to one of the above arguments) will print out every target as it is built, and the dependency which caused it to be built.
307 - Running ``make -p`` prints a (very verbose) summary of every generated target in each makefile.
309 For more debugging tips and general make information, see the `GNU Make Manual`.
311 Overriding Parts of the Project
312 -------------------------------
317 For components that have build requirements that must be evaluated in the top-level
318 project make pass, you can create a file called ``Makefile.projbuild`` in the
319 component directory. This makefile is included when ``project.mk`` is evaluated.
321 For example, if your component needs to add to CFLAGS for the entire
322 project (not just for its own source files) then you can set
323 ``CFLAGS +=`` in Makefile.projbuild.
325 ``Makefile.projbuild`` files are used heavily inside esp-idf, for defining project-wide build features such as ``esptool.py`` command line arguments and the ``bootloader`` "special app".
327 Note that ``Makefile.projbuild`` isn't necessary for the most common component uses - such as adding include directories to the project, or LDFLAGS to the final linking step. These values can be customised via the ``component.mk`` file itself. See `Optional Project-Wide Component Variables` for details.
329 Take care when setting variables or targets in this file. As the values are included into the top-level project makefile pass, they can influence or break functionality across all components!
334 This is an equivalent to `Makefile.projbuild` for `component configuration` KConfig files. If you want to include
335 configuration options at the top-level of menuconfig, rather than inside the "Component Configuration" sub-menu, then these can be defined in the KConfig.projbuild file alongside the ``component.mk`` file.
337 Take care when adding configuration values in this file, as they will be included across the entire project configuration. Where possible, it's generally better to create a KConfig file for `component configuration`.
340 Example Component Makefiles
341 ---------------------------
343 Because the build environment tries to set reasonable defaults that will work most
344 of the time, component.mk can be very small or even empty (see `Minimal Component Makefile`). However, overriding `component variables` is usually required for some functionality.
346 Here are some more advanced examples of ``component.mk`` makefiles:
349 Adding source directories
350 ^^^^^^^^^^^^^^^^^^^^^^^^^
352 By default, sub-directories are ignored. If your project has sources in sub-directories
353 instead of in the root of the component then you can tell that to the build
354 system by setting ``COMPONENT_SRCDIRS``::
356 COMPONENT_SRCDIRS := src1 src2
358 This will compile all source files in the src1/ and src2/ sub-directories
361 Specifying source files
362 ^^^^^^^^^^^^^^^^^^^^^^^
364 The standard component.mk logic adds all .S and .c files in the source
365 directories as sources to be compiled unconditionally. It is possible
366 to circumvent that logic and hard-code the objects to be compiled by
367 manually setting the ``COMPONENT_OBJS`` variable to the name of the
368 objects that need to be generated::
370 COMPONENT_OBJS := file1.o file2.o thing/filea.o thing/fileb.o anotherthing/main.o
371 COMPONENT_SRCDIRS := . thing anotherthing
373 Note that ``COMPONENT_SRCDIRS`` must be set as well.
375 Adding conditional configuration
376 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
378 The configuration system can be used to conditionally compile some files
379 depending on the options selected in ``make menuconfig``:
383 config FOO_ENABLE_BAR
384 bool "Enable the BAR feature."
386 This enables the BAR feature of the FOO component.
390 COMPONENT_OBJS := foo_a.o foo_b.o
393 COMPONENT_OBJS += foo_bar.o foo_bar_interface.o
396 See the `GNU Make Manual` for conditional syntax that can be used use in makefiles.
399 Source Code Generation
400 ^^^^^^^^^^^^^^^^^^^^^^
402 Some components will have a situation where a source file isn't
403 supplied with the component itself but has to be generated from
404 another file. Say our component has a header file that consists of the
405 converted binary data of a BMP file, converted using a hypothetical
406 tool called bmp2h. The header file is then included in as C source
407 file called graphics_lib.c::
409 COMPONENT_EXTRA_CLEAN := logo.h
411 graphics_lib.o: logo.h
413 logo.h: $(COMPONENT_PATH)/logo.bmp
417 In this example, graphics_lib.o and logo.h will be generated in the
418 current directory (the build directory) while logo.bmp comes with the
419 component and resides under the component path. Because logo.h is a
420 generated file, it needs to be cleaned when make clean is called which
421 why it is added to the COMPONENT_EXTRA_CLEAN variable.
423 Cosmetic Improvements
424 ^^^^^^^^^^^^^^^^^^^^^
426 Because logo.h is a generated file, it needs to be cleaned when make
427 clean is called which why it is added to the COMPONENT_EXTRA_CLEAN
430 Adding logo.h to the ``graphics_lib.o`` dependencies causes it to be
431 generated before ``graphics_lib.c`` is compiled.
433 If a a source file in another component included ``logo.h``, then this
434 component's name would have to be added to the other component's
435 ``COMPONENT_DEPENDS`` list to ensure that the components were built
438 Embedding Binary Data
439 ^^^^^^^^^^^^^^^^^^^^^
441 Sometimes you have a file with some binary or text data that you'd like to make available to your component - but you don't want to reformat the file as C source.
443 You can set a variable COMPONENT_EMBED_FILES in component.mk, giving the names of the files to embed in this way::
445 COMPONENT_EMBED_FILES := server_root_cert.der
447 Or if the file is a string, you can use the variable COMPONENT_EMBED_TXTFILES. This will embed the contents of the text file as a null-terminated string::
449 COMPONENT_EMBED_TXTFILES := server_root_cert.pem
451 The file's contents will be added to the .rodata section in flash, and are available via symbol names as follows::
453 extern const uint8_t server_root_cert_pem_start[] asm("_binary_server_root_cert_pem_start");
454 extern const uint8_t server_root_cert_pem_end[] asm("_binary_server_root_cert_pem_end");
456 The names are generated from the full name of the file, as given in COMPONENT_EMBED_FILES. Characters /, ., etc. are replaced with underscores. The _binary prefix in the symbol name is added by objcopy and is the same for both text and binary files.
458 For an example of using this technique, see examples/04_https_request - the certificate file contents are loaded from the text .pem file at compile time.
461 Fully Overriding The Component Makefile
462 ---------------------------------------
464 Obviously, there are cases where all these recipes are insufficient for a
465 certain component, for example when the component is basically a wrapper
466 around another third-party component not originally intended to be
467 compiled under this build system. In that case, it's possible to forego
468 the esp-idf build system entirely by setting COMPONENT_OWNBUILDTARGET and
469 possibly COMPONENT_OWNCLEANTARGET and defining your own targets named ``build`` and ``clean`` in ``component.mk``
470 target. The build target can do anything as long as it creates
471 $(COMPONENT_LIBRARY) for the project make process to link into the app binary.
473 (Actually, even this is not strictly necessary - if the COMPONENT_ADD_LDFLAGS variable
474 is set then the component can instruct the linker to link other binaries instead.)
477 .. _esp-idf-template: https://github.com/espressif/esp-idf-template
478 .. _GNU Make Manual: https://www.gnu.org/software/make/manual/make.html
479 .. _[_f1]: Actually, some components in esp-idf are "pure configuration" components that don't have a component.mk file, only a Makefile.projbuild and/or Kconfig.projbuild file. However, these components are unusual and most components have a component.mk file.
projects / esp-idf / blob