Build RISC-V Toolchain

This doc lists the common config settings to build the RISC-V toolchain. It generally involves two parts of the toolchain: GCC to build the headers and libraries, and LLVM to build the compiler, linker, and utility tools.

Prerequisites

Your host machine needs to have the following packages installed, which are already part of the Shodan prerequisite pacakges

  • CMake (>= 3.13.4)
  • Ninja
  • Clang

The source code of the toolchain is at

Build RISC-V Linux toolchain (64-bit)

Build GCC

$ mkdir -p <GCC_BUILD_PATH>
$ cd <GCC_BUILD_PATH>
$ <GCC_SRC_PATH>/configure \
  --srcdir=<GCC_SRC_PATH> \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --with-arch=rv64gc \
  --with-abi=lp64d \
  --with-cmodel=medany
$ make -C <GCC_BUILD_PATH> linux

Notice Linux requires the full general CPU extension support, i.e., rv64imafdc, and the ABI also needs to support hard double-float modules. For 32-bit Linux, build the toolchain with the flags of --with-arch=rv32gc --with-abi=ilp32d.

Build LLVM

$ cmake -B <LLVM_BUILD_PATH> \
    -DCMAKE_INSTALL_PREFIX=<TOOLCHAIN_OUT_DIR> \
    -DCMAKE_C_COMPILER=clang  -DCMAKE_CXX_COMPILER=clang++ \
    -DCMAKE_BUILD_TYPE=Release \
    -DLLVM_TARGETS_TO_BUILD="RISCV" \
    -DLLVM_ENABLE_PROJECTS="clang"  \
    -DLLVM_DEFAULT_TARGET_TRIPLE="riscv64-unknown-linux-gnu" \
    -DLLVM_INSTALL_TOOLCHAIN_ONLY=On \
    -DDEFAULT_SYSROOT=../sysroot \
    -G Ninja \
    <LLVM_SRC_PATH>/llvm
$ cmake --build  <LLVM_BUILD_PATH> --target install

For 32-bit, change the LLVM target triple to riscv32-unknown-linux-gnu.

Build RISC-V bare-metal toolchain (32-bit)

Build 32-bit GCC

$ mkdir -p <GCC_BUILD_PATH>
$ cd <GCC_BUILD_PATH>
$ <GCC_SRC_PATH>/configure \
  --srcdir=<GCC_SRC_PATH> \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --with-arch=rv32i2p0mf2p0 \
  --with-abi=ilp32 \
  --with-cmodel=medany
$ make -C <GCC_BUILD_PATH> newlib

Notice for bare-metal newlib there's no hard constraints on CPU feature and ABI support. However, LLVM for bare-metal only supports soft-float modules, so the GCC ABI setting needs to match that. Also, the ISA version needs to be specified, since binuils ISA supports 20191213 spec and LLVM is at v2.2 spec

Build 32-bit LLVM

$ cmake -B <LLVM_BUILD_PATH> \
    -DCMAKE_INSTALL_PREFIX=<TOOLCHAIN_OUT_DIR> \
    -DCMAKE_C_COMPILER=clang  -DCMAKE_CXX_COMPILER=clang++ \
    -DCMAKE_BUILD_TYPE=Release \
    -DLLVM_TARGETS_TO_BUILD="RISCV" \
    -DLLVM_ENABLE_PROJECTS="clang"  \
    -DLLVM_DEFAULT_TARGET_TRIPLE="riscv32-unknown-elf" \
    -DLLVM_INSTALL_TOOLCHAIN_ONLY=On \
    -DDEFAULT_SYSROOT=../riscv32-unknown-elf \
    -G Ninja \
    <LLVM_SRC_PATH>/llvm
$ cmake --build  <LLVM_BUILD_PATH> --target install

Build compiler-rt

This should not be necessary for the Shodan usage, but in case the compiler-rt builtins is required in the project (instead of using libgcc), it can be built with the additional commands:

$ export PATH=<TOOLCHAIN_OUT_DIR>/bin:${PATH}
$ cmake -B <LLVM_BUILD_PATH>/compiler-rt \
  -DCMAKE_INSTALL_PREFIX=$PREFIX \
  -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY \
  -DCMAKE_AR=<TOOLCHAIN_OUT_DIR>/bin/llvm-ar \
  -DCMAKE_NM=<TOOLCHAIN_OUT_DIR>/bin/llvm-nm \
  -DCMAKE_RANLIB=<TOOLCHAIN_OUT_DIR>/bin/llvm-ranlib \
  -DCMAKE_C_FLAGS="-march=rv32i2p0mf2p0v1p0" \
  -DCMAKE_ASM_FLAGS="-march=rv32i2p0mf2p0v1p0" \
  -DCMAKE_C_COMPILER=<TOOLCHAIN_OUT_DIR>/bin/clang \
  -DCMAKE_C_COMPILER_TARGET=riscv32-unknown-elf \
  -DCMAKE_ASM_COMPILER_TARGET=riscv32-unknown-elf \
  -DCOMPILER_RT_OS_DIR="clang/14.0.0/lib" \
  -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \
  -DCOMPILER_RT_BUILD_BUILTINS=ON \
  -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
  -DCOMPILER_RT_BUILD_XRAY=OFF \
  -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
  -DCOMPILER_RT_BUILD_MEMPROF=OFF \
  -DCOMPILER_RT_BUILD_PROFILE=OFF \
  -DCOMPILER_RT_BAREMETAL_BUILD=ON \
  -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
  -DLLVM_CONFIG_PATH=<LLVM_BUILD_PATH>/bin/llvm-config \
  -DCMAKE_C_FLAGS="-march=rv32i2p0mf2p0v1p0 -mno-relax" \
  -DCMAKE_ASM_FLAGS="-march=rv32i2p0mf2p0v1p0 -mno-relax" \
  -G "Ninja" <LLVM_SRC_PATH>/compiler-rt
$ cmake --build <LLVM_BUILD_PATH>/compiler-rt --target install

Build newlib

Even with compiler_rt to replace libgcc, we still need to build libc, libm, and libgloss as the toolchain prebuilts. They can also be built with clang The source code is at https://github.com/riscv/riscv-newlib

NOTE: The GCC utility tools needs to be built first.

$ mkdir -p <NEWLIB_BUILD_PATH>
$ cd <NEWLIB_BUILD_PATH>
$ <NEWLIB_SRC_PATH>/configure \
  --target=riscv32-unknown-elf \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --enable-newlib-io-long-double \
  --enable-newlib-io-long-long \
  --enable-newlib-io-c99-formats \
  --enable-newlib-register-fini \
  CC_FOR_TARGET=clang \
  CXX_FOR_TARGET=clang++ \
  CFLAGS_FOR_TARGET="-march=rv32i2p0mf2p0v1p0 -O2 -D_POSIX_MODE -mno-relax" \
  CXXFLAGS_FOR_TARGET="-march=rv32i2p0mf2p0v1p0 -O2 -D_POSIX_MODE -mno-relax"
$ make -j32
$ make install

The newlib nano spec needs to be built separatedly and then merged with newlib. GNU top-level Makefile lists the flags to build nano and the merging script.

Test toolchain

Run

<TOOLCHAIN_OUT_DIR>/bin/<arch>-<os>-<abi>-gcc -v

to see the supported ABIs, architectures, library paths, etc.

Try to compile a simple c code (copied from CMake's package content, e.g., /usr/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake)

#ifdef __cplusplus
# error "The CMAKE_C_COMPILER is set to a C++ compiler"
#endif
#if defined(__CLASSIC_C__)
int main(argc, argv)
  int argc;
  char* argv[];
#else
int main(int argc, char* argv[])
#endif
{ (void)argv; return argc-1;}

To build (32-bit bare-metal example)

$<TOOLCHAIN_OUT_DIR>/bin/clang -c testCCompiler.c -O --target=riscv32
$<TOOLCHAIN_OUT_DIR>/bin/riscv32-unknown-elf-gcc testCCompiler.o -o testCCompiler -march=rv32gc -mabi=ilp32

You can also use readelf to inspect the object file before building the binary to see if the architecture and ABI match.