Sysroots

This document explains the sysroot system used by the Sol3 SDK for cross-platform builds and runtime consistency.

What is a Sysroot?

A sysroot is a directory that acts as the logical root filesystem during compilation and linking. It contains all the headers, libraries, and runtime dependencies needed to build and run applications for a specific target architecture.

In traditional development, compilers search for libraries in system directories like /usr/lib and /usr/include. With sysroots, the compiler searches in <sysroot>/usr/lib and <sysroot>/usr/include instead, isolating the build from the host system.

Purpose and Benefits

Build Isolation - The compiler finds dependencies only within the sysroot, preventing accidental linking to incompatible host libraries. This ensures that x64 builds use x64 libraries and ARM64 builds use ARM64 libraries, even when building on the same machine.

Cross-Compilation Support - An x64 development machine can build ARM64 binaries by using an ARM64 sysroot. The cross-compiler links against ARM64 libraries in the sysroot, producing binaries that run on ARM64 hardware.

Runtime Consistency - The sysroot used during development matches the filesystem in the deployment container. Applications tested in the devcontainer behave identically in production.

Reproducible Builds - Sysroots provide a fixed set of dependencies, eliminating “works on my machine” issues caused by different host system configurations.

What’s in the Sysroot

System Libraries:

  • C standard library (libc.so)

  • C++ standard library (libstdc++.so)

  • Additional libraries not managed via vcpkg (libm.so, libpthread.so, etc.)

  • OpenCL (x64: ocl-icd; arm64: qti-adreno) and GBM, which the toolchain and FindOpenCL resolve from the sysroot

Not Included:

  • Build tools (compilers, CMake)

  • Development utilities (debuggers, profilers)

  • Third-party libraries from vcpkg (these are in vcpkg’s install directory)

  • Source code

Architecture-Specific Sysroots

The Sol3 SDK provides separate sysroots for each target architecture.

Sysroot Locations

Within the devcontainer, sysroots are located at:

/opt/sysroots/
├── x64/           # x86_64 runtime environment
│   ├── lib/
│   ├── usr/
│   │   ├── lib/
│   │   └── include/
│   └── ...
└── arm64/         # ARM64/AArch64 runtime environment
    ├── lib/
    ├── usr/
    │   ├── lib/
    │   └── include/
    └── ...

Architecture Selection

Each CMake preset selects a toolchain file (via VCPKG_CHAINLOAD_TOOLCHAIN_FILE):

  • x64 preset → x64-sol3-clang toolchain → /opt/sysroots/x64/

  • arm64 preset → arm64-sol3-clang toolchain → /opt/sysroots/arm64/

The toolchain file (not the preset itself) sets CMake’s CMAKE_SYSROOT, defaulting to /opt/sysroots/<arch> but overridable via the SYSROOT_PATH environment variable. CMAKE_SYSROOT tells the compiler and linker where to find dependencies.

Build-Time Usage

During compilation and linking, the sysroot provides headers and libraries for the target architecture.

Compiler Integration

The CMake toolchain configures the compiler to use the sysroot:

# Set by the toolchain file (default; overridable via SYSROOT_PATH)
set(CMAKE_SYSROOT /opt/sysroots/arm64)

# Compiler searches for headers in:
# ${CMAKE_SYSROOT}/usr/include
# ${CMAKE_SYSROOT}/usr/local/include

# Linker searches for libraries in:
# ${CMAKE_SYSROOT}/lib
# ${CMAKE_SYSROOT}/usr/lib

Runtime Usage

At runtime, the dynamic linker must find shared libraries in the sysroot.

Dynamic Linker Configuration

The sol3_activate_sysroot script configures the runtime environment:

# Activate x64 sysroot
sol3_activate_sysroot x64 ./out/build/x64/bin/example_app

This sets environment variables:

LD_LIBRARY_PATH=/opt/sysroots/x64/lib:/opt/sysroots/x64/usr/lib:/opt/sysroots/x64/usr/local/lib:/opt/sysroots/x64/usr/lib/x86_64-linux-gnu
QEMU_LD_PREFIX=/opt/sysroots/x64

The dynamic linker (ld.so) uses LD_LIBRARY_PATH to locate shared libraries when launching the application. QEMU_LD_PREFIX lets the qemu binfmt handler find the loader and libraries when running ARM64 binaries on an x64 host.

ctest --preset <arch> applies the same environment automatically (via the test-<arch>-sysroot test presets), so running tests through the preset needs no manual activation.

Why Runtime Activation is Required

Without sysroot activation, the dynamic linker searches the host system’s library paths. This causes errors when the host lacks required libraries or provides incompatible versions.

Example without activation:

$ ./out/build/x64/bin/example_app
./example_app: error while loading shared libraries: libgbm.so.1: cannot open shared object file: No such file or directory

Example with activation:

$ sol3_activate_sysroot x64 ./out/build/x64/bin/example_app
# Application runs successfully

Deployment Container Runtime

In the deployment container, the sysroot is the entire filesystem. The application container is built from the sysroot stage, so all libraries are in standard locations (/lib, /usr/lib). No runtime activation is needed.