Build System
The build system downloads and creates several files that you may want to know about. There are also several make
commands mentioned below, and a few extras that you may find useful. Here's a quick summary. All file paths are relative to your redox
base directory.
- Build System Organization
- GNU Make Commands
- Environment Variables
- Scripts
- Component Separation
- Crates
- Pinned commits
- Git auto-checkout
- Update The Build System
- Update relibc
- Configuration
- Cross-Compilation
- Build Phases
Build System Organization
Root Folder
podman_bootstrap.sh
- The script used to configure the Podman buildnative_bootstrap.sh
- The script used to configure the Native buildMakefile
- The main Makefile of the build system, it loads all the other Makefiles..config
- Where you override your build system settings. It is loaded by the Makefile (it is ignored bygit
).
GNU Make Configuration
mk/config.mk
- The build system settings are here. You can override these settings in your.config
, don't change them here to avoid conflicts inmake pull
.mk/*.mk
- The rest of the Makefiles. You should not need to change them.
Podman Configuration
podman/redox-base-containerfile
- The file used to create the image used by the Podman build. The installation of Ubuntu packages needed for the build is done here. See Adding Packages to the Build if you need to add additional Ubuntu packages.
Filesystem Configuration
config
- This folder contains all filesystem configurations.config/*.toml
- Filesystem templates used by the CPU target configurations (a template can use other template to reduce duplication)config/your-cpu-arch/your-config.toml
- The filesystem configuration of the QEMU image to be built, e.g.config/x86_64/desktop.toml
.config/your-cpu-arch/server.toml
- The variant with system components (without Orbital) and some important tools. Aimed for servers, low-end computers, testers and developers.
(Try this config if you have boot problems on QEMU or real hardware).
config/your-cpu-arch/desktop.toml
- The variant with system components, the Orbital desktop environment and some important programs (this is the default configuration of the build system). Aimed for end-users, gamers, testers and developers.config/your-cpu-arch/dev.toml
- The variant with development tools included. Aimed for developers.config/your-cpu-arch/demo.toml
- The variant with a complete system and optional programs and games. Aimed for end-users, gamers, testers and developers.config/your-cpu-arch/desktop-minimal.toml
- The minimaldesktop
variant for low-end computers and embedded hardware. Aimed for servers, low-end computers, embedded hardware and developers.config/your-cpu-arch/minimal.toml
- The variant without network support and Orbital. Aimed for low-end computers, embedded hardware, testers and developers.config/your-cpu-arch/minimal-net.toml
- The variant without Orbital and tools. Aimed for low-end computers, embedded hardware, testers and developers.config/your-cpu-arch/resist.toml
- The variant with theresist
POSIX test suite. Aimed for developers.config/your-cpu-arch/acid.toml
- The variant with theacid
general-purpose test suite. Aimed for developers.config/your-cpu-arch/ci.toml
- The continuous integration variant, recipes added here become packages on the build server. Aimed for packagers and developers.config/your-cpu-arch/jeremy.toml
- The build of Jeremy Soller (creator/BDFL of Redox) with the recipes that he is testing at the moment.
Cookbook
prefix/*
- Tools used by the Cookbook system. They are normally downloaded during the first system build.
(If you are having a problem with the build system, you can remove the prefix
directory and it will be recreated during the next build)
cookbook/*
- Part of the Cookbook system, these scripts and tools will build your recipes.cookbook/repo
- Contains all packaged recipes.cookbook/recipes/recipe-name
- A recipe (software port) directory (represented asrecipe-name
), this directory holds therecipe.toml
file.cookbook/recipes/recipe-name/recipe.toml
- The recipe configuration file, this configuration contains instructions for downloading Git repositories or tarballs, then creating executables or other files to include in the Redox filesystem. Note that a recipe can contain dependencies that cause other recipes to be built, even if the dependencies are not otherwise part of your Redox build.
(To learn more about the recipe system read this page)
cookbook/recipes/recipe-name/recipe.sh
- The old recipe configuration format (can't be used as dependency of a recipe with a TOML syntax).cookbook/recipes/recipe-name/source.tar
- The tarball of the recipe (renamed).cookbook/recipes/recipe-name/source
- The directory where the recipe source is extracted or downloaded.cookbook/recipes/recipe-name/target
- The directory where the recipe binaries are stored.cookbook/recipes/recipe-name/target/${TARGET}
- The directory for the recipes binaries of the CPU architecture (${TARGET}
is the environment variable of your CPU architecture).cookbook/recipes/recipe-name/target/${TARGET}/build
- The directory where the recipe build system run its commands.cookbook/recipes/recipe-name/target/${TARGET}/stage
- The directory where recipe binaries go before the packaging, aftermake all
,make rebuild
andmake image
the installer will extract the recipe package on the QEMU image, generally at/usr/bin
or/usr/lib
in a Redox filesystem hierarchy.cookbook/recipes/recipe-name/target/${TARGET}/sysroot
- The folder where recipe build dependencies (libraries) goes, for example:library-name/src/example.c
cookbook/recipes/recipe-name/target/${TARGET}/stage.pkgar
- Redox package file.cookbook/recipes/recipe-name/target/${TARGET}/stage.sig
- Signature for thetar
package format.cookbook/recipes/recipe-name/target/${TARGET}/stage.tar.gz
- Legacytar
package format, produced for compatibility reasons as we are working to make the package manager use thepkgar
format.cookbook/recipes/recipe-name/target/${TARGET}/stage.toml
- Contains the runtime dependencies of the package and is part of both package formats.
Build System Files
build
- The directory where the build system will place the final image. Usuallybuild/$(ARCH)/$(CONFIG_NAME)
, e.g.build/x86_64/desktop
.build/your-cpu-arch/your-config/harddrive.img
- The Redox image file, to be used by QEMU or VirtualBox for virtual machine execution on a Unix-like host.build/your-cpu-arch/your-config/livedisk.iso
- The Redox bootable image file, to be used on real hardware for testing and possible installation.build/your-cpu-arch/your-config/fetch.tag
- An empty file that, if present, tells the build system that the downloading of recipe sources is done.build/your-cpu-arch/your-config/repo.tag
- An empty file that, if present, tells the build system that all recipes required for the Redox image have been successfully built. The build system will not check for changes to your code when this file is present. Usemake rebuild
to force the build system to check for changes.build/podman
- The directory where Podman Build places the container user's home directory, including the container's Rust installation. Usemake container_clean
to remove it. In some situations, you may need to remove this directory manually, possibly with root privileges.build/container.tag
- An empty file, created during the first Podman build, so a Podman build knows when a reusable Podman image is available. Usemake container_clean
to force a rebuild of the Podman image on your nextmake rebuild
.
GNU Make Commands
You can combine make
commands, but order is significant. For example, make r.games image
will build the games
recipe and create a new Redox image, but make image r.games
will make the Redox image before the recipe building, thus the new recipe binary will not be included on your Redox filesystem.
Build System
make pull
- Update the source code of the build system without building.make all
- Builds the entire system, checking for changes and only building as required. Only use this for the first build. If the system was successfully built previously, this command may reportNothing to be done for 'all'
, even if some recipes have changed. Usemake rebuild
instead.make rebuild
- Update all binaries from recipes with source code changes (it don't detect changes on the Redox toolchain), it should be your normalmake
target.make prefix
- Download the Rust/GCC forks and build relibc (it update the relibc binary with source code changes after themake pull
,touch relibc
andmake prefix
commands).make fstools
- Build the image builder (installer), Cookbook and RedoxFS (aftertouch installer
ortouch redoxfs
).make fstools_clean
- Clean the image builder, Cookbook and RedoxFS binaries.make fetch
- Update recipe sources, according to each recipe, without building them. Only the recipes that are included in your(CONFIG_NAME).toml
are downloaded. Does nothing if$(BUILD)/fetch.tag
is present. You won't need this.make clean
- Clean all recipe binaries (Note thatmake clean
may require some tools to be built).make unfetch
- Clean all recipe sources.make distclean
- Clean all recipe sources and binaries.make repo
- Package the recipe binaries, according to each recipe. Does nothing if$(BUILD)/repo.tag
is present. You won't need this.make live
- Creates a bootable image,build/livedisk.iso
. Recipes are not usually rebuilt.make popsicle
- Flash the Redox bootable image on your USB device using the Popsicle tool (the program executable must be present on your shell$PATH
environment variable, you can get the executable by extracting the AppImage, installing from the package manager or building from source)make env
- Creates a shell with the build environment initialized. If you are using Podman Build, the shell will be inside the container, and you can use it to debug build issues such as missing packages.make container_shell
- Open the GNU Bash shell of the Podman container as the active shell of your terminal, it's logged as thepodman
user withoutroot
privileges.make container_su
- Open the GNU Bash shell of the Podman container as the active shell of your terminal, it's logged as theroot
user.make container_clean
- This will discard images and other files created by Podman.make container_touch
- If you have removed the filebuild/container.tag
, but the container image is still usable, this will recreate thecontainer.tag
file and avoid rebuilding the container image.make container_kill
- If you have started a build using Podman Build, and you want to stop it,Ctrl-C
may not be sufficient. Use this command to terminate the most recently created container.
Recipes
-
make f.recipe-name
- Download the recipe source. -
make r.recipe-name
- Build a single recipe, checking if the recipe source has changed, and creating the executable, etc. e.g.make r.games
.The package is built even if it is not in your filesystem configuration.
(This command will continue where you stopped the build process, it's useful to save time if you had a compilation error and patched a crate)
-
make c.recipe-name
- Clean the recipe binaries. -
make u.recipe-name
- Clean the recipe source code. -
make cr.recipe-name
- A shortcut formake c.recipe r.recipe
. -
make ucr.recipe-name
- A shortcut formake u.recipe c.recipe r.recipe
.
QEMU/VirtualBox
make qemu
- If abuild/harddrive.img
file exists, QEMU will run using that image. If you want to force a rebuild first, usemake rebuild qemu
. Sometimesmake qemu
will detect changes and rebuild, but this is not typical. If you are interested in a particular combination of QEMU command line options, have a look throughmk/qemu.mk
.make qemu gpu=no
- Start QEMU without a GUI (Orbital is disabled).make qemu gpu=virtio
- Start QEMU with the VirtIO GPU driver.make qemu kvm=no
- Start QEMU without the Linux KVM acceleration.make qemu iommu=no
- Start QEMU without IOMMU.make qemu audio=no
- Disable all sound drivers.make qemu usb=no
- Disable all USB drivers.make qemu uefi=yes
- Enable the UEFI boot loader (it supports more screen resolutions).make qemu live=yes
- Fully load the Redox image to RAM.make qemu disk=nvme
- Boot Redox from a NVMe interface (SSD emulation).make qemu disk=usb
- Boot Redox from a virtual USB device.make qemu disk=cdrom
- Boot Redox from a virtual CD-ROM disk.make qemu option1=string option2=string
- Cumulative QEMU options is supported.make image
- Builds a new QEMU image,build/harddrive.img
, without checking if any recipes have changed. It can save you some time if you are just updating one recipe withmake r.recipe-name
.make gdb
- Connectsgdb
to the Redox image in QEMU. Join us on chat if you want to use this.make mount
- Mounts the Redox image as a filesystem at$(BUILD)/filesystem
. Do not use this if QEMU is running, and remember to usemake unmount
as soon as you are done. This is not recommended, but if you need to get a large file onto or off of your Redox image, this is available as a workaround.make unmount
- Unmounts the Redox image filesystem. Use this as soon as you are done withmake mount
, and do not start QEMU until this is done.make virtualbox
- The same asmake qemu
, but for VirtualBox (it requires the VirtualBox service to be running, runsystemctl status vboxdrv.service
to verify orakmods; systemctl restart vboxdrv.service
to enable on systems using systemd).
Environment Variables
$(BUILD)
- Represents thebuild
folder.$(ARCH)
- Represents the CPU architecture folder atbuild
.${TARGET}
- Represents the CPU architecture folder atcookbook/recipes/recipe-name/target
.$(CONFIG_NAME)
- Represents your filesystem configuration folder atbuild/your-cpu-arch
.
We recommend that you use these variables with the "
symbol to clean any spaces on the path, spaces are interpreted as command separators and will break the commands.
Example:
"${VARIABLE_NAME}"
If you have a folder inside the variable folder you can call it with:
"${VARIABLE_NAME}"/folder-name
Or
"${VARIABLE_NAME}/folder-name"
Scripts
You can use these scripts to perform actions not implemented as make
commands in the build system.
- To run a script use the following command:
scripts/script-name.sh input-text
The "input-text" is the word used by the script.
Changelog
scripts/changelog.sh
- Show the changelog of all Redox components.
Recipe Files
Show all files installed by a recipe.
scripts/find-recipe.sh recipe-name
Recipe Categories
Run make
options on some recipe category.
scripts/category.sh -x category-name
Where x
is your make
option, it can be f
, r
, c
, u
, cr
, ucr
, uc
or ucf
.
Include Recipes
Create a list with recipe-name = {} #TODO
for quick testing of WIP recipes.
scripts/include-recipes.sh "TODO.text"
You will insert the text after the #TODO
in the text
part, it can be found on the recipe.toml
file of the recipe.
If you want to add recipes to the ci.toml
filesystem configuration to be available on the package build server, the recipe names must be sorted in alphabetical order, to do this from the output of this script use the following command:
scripts/include-recipes.sh "TODO.text" | sort
Recipe Analysis
Show the folders and files on the stage
and sysroot
folders of some recipe (to identify packaging issues or violations).
scripts/show-package.sh recipe-name
Recipe Commit Hash
Show the current Git branch and commit of the recipe source.
scripts/commit-hash.sh recipe-name
Package Size
Show the package size of the recipes (stage.pkgar
and stage.tar.gz
), it must be used by package maintainers to enforce the library linking size policy.
scripts/pkg-size.sh recipe-name
Recipe Location
Show the location of the written recipe.
scripts/recipe-path.sh recipe-name
Recipe Match
Search some text inside the recipe.toml
of some recipe.
scripts/recipe-match.sh recipe-name
Print Recipe
Show the content of the recipe configuration.
(Require bat
and ripgrep
installed, run cargo install bat ripgrep
to install)
scripts/print-recipe.sh recipe-name
Recipe Executables
List the recipe executables to find duplicates and conflicts.
- By default the script will only verify duplicates, if the
-a
option is used it will print the executable names of all compiled recipes - The
-arm64
option will show the ARM64 recipe executables - The
-i686
option will show the i686 recipe executables
scripts/executables.sh
Cargo Update
Download the recipe source and run cargo update
scripts/cargo-update.sh recipe-name
Dual Boot
scripts/dual-boot.sh
- Install Redox in the free space of your storage device and add a boot entry (if you are using the systemd-boot boot loader).
Ventoy
scripts/ventoy.sh
- Create and copy the Redox bootable image to an Ventoy-formatted device.
Recipe Debugging (Rust)
scripts/backtrace.sh
- Allow the user to copy a Rust backtrace from Redox and retrieve the symbols (use the-h
option to show the "Usage" message).
Component Separation
relibc
- The cross-compiled recipes will link to the relibc of this folder (build system submodule)redoxfs
- The FUSE driver of RedoxFS (build system submodule, to run on Linux)cookbook/recipes/relibc
- The relibc recipe to be installed inside of Redox for static or dynamic linking of binaries (for native compilation)cookbook/recipes/redoxfs
- The RedoxFS user-space daemon that run inside of Redox (recipe)
Crates
Some Redox projects have crates on crates.io
, thus they use a version-based development, if some change is sent to their repository they need to release a new version on crates.io
, it will have some delay.
Current projects with crates
- libredox
- redox_syscall
- redox-path
- redox-scheme
- redoxfs
- redoxer
- redox_installer
- redox-kprofiling
- redox-users
- redox-buffer-pool
- redox_log
- redox_termios
- redox-daemon
- redox_event
- redox_event_update
- redox_pkgutils
- redox_uefi
- redox_uefi_alloc
- redox_dmi
- redox_hwio
- redox_intelflash
- redox_liner
- redox_uefi_std
- ralloc
- orbclient
- orbclient_window_shortcuts
- orbfont
- orbimage
- orbterm
- orbutils
- slint_orbclient
- ralloc_shim
- ransid
- gitrepoman
- pkgar
- pkgar-core
- pkgar-repo
- termion
- reagent
- gdb-protocol
- orbtk
- orbtk_orbclient
- orbtk-render
- orbtk-shell
- orbtk-tinyskia
Manual patching
If you don't want to wait a new release on crates.io
, you can patch the crate temporarily by fetching the version you need from GitLab and changing the crate version in Cargo.toml
to crate-name = { path = "path/to/crate" }
.
Pinned commits
The build system pin the last working commit of the submodules, if some submodule is broken because of some commit, the pinned commit avoid the fetch of this broken commit, thus pinned commits increase the development stability (broken changes aren't passed for developers and testers).
(When you run make pull
the build system update the submodule folders based on the last pinned commit)
Current pinned submodules
cookbook
installer
redoxfs
relibc
rust
Manual submodule update
Whenever a fix or new feature is merged on the submodules, the upstream build system must update the commit hash, to workaround this you can run git pull
on the folder of the submodule directly, example:
make pull
cd submodule-folder-name
git checkout master
git pull
cd ..
Git auto-checkout
The make rebuild
and make r.recipe
commands will Git checkout (change the active branch) of the recipe source to master
(only recipes that fetch Git repositories are affected, thus all Redox components).
If you are working in a separated branch on the recipe source you can't build your changes, to avoid this comment out the [source]
and git =
fields from your recipe.toml
:
#[source]
#git = "some-repository-link"
Submodules
The make pull
command will Git checkout the submodules active branches to master
and pin a commit in HEAD
, if you are working on the build system submodules don't run this command, to keep the build system using your changes.
To update the build system while you have out-of-tree changes in some submodule, run these commands:
- This command will only update the root build system files
git pull
- Now you need to manually update each submodule folder without your changes
cd submodule-name
git checkout master
git pull
cd ..
Update The Build System
This is the recommended way to update your build system/recipe sources and binaries.
make pull rebuild
(If the make pull
command download new commits of the relibc
submodule, you will need to run the commands of the Update relibc section)
Some new changes will require a complete rebuild (you will need to read the Dev room in our chat to know if some big MR was merged and run the make clean all
command) or a new build system copy (run the bootstrap.sh script again or run the commands of this section), but the make pull rebuild
command is enough for most cases.
Update relibc
An outdated relibc copy can contain bugs (already fixed on recent commits) or missing APIs, to update the relibc sources and build it, run:
make pull
touch relibc
make prefix
All recipes
To pass the new relibc changes for all recipes (programs are the most common case) you will need to rebuild all recipes, unfortunately it's not possible to use make rebuild
because it can't detect the relibc changes to trigger a complete rebuild.
To clean all recipe binaries and trigger a complete rebuild, run:
make clean all
One recipe
To pass the new relibc changes to one recipe, run:
make cr.recipe-name
Configuration
You can find the global settings on this page.
Format
The Redox configuration files use the TOML format, it has a very easy syntax and is very flexbile.
You can see what TOML supports on this website.
Filesystem Customization
Read this section.
Cross-Compilation
The Redox build system is an example of cross-compilation. The Redox toolchain runs on Linux, and produces Redox executables. Anything that is installed with your package manager is just part of the toolchain and does not go on Redox.
In the background, the make all
command downloads the Redox toolchain to build all recipes (patched forks of rustc, GCC and LLVM).
If you are using Podman, the podman_bootstrap.sh
script will download an Ubuntu container and make all
will install the Redox toolchain, all recipes will be compiled in the container.
The recipes produce Redox-specific executables. At the end of the build process, these executables are installed inside the QEMU image.
The relibc
(Redox C Library) provides the Redox system calls to any software.
Build Phases
Every build system command/script has phases, read this page to know them.