BuildRiscVToolchain.md

# Build RISC-V Toolchain

This doc lists the common config settings to build the RISC-V toolchain. It
generally involves two parts of the toolchain: GCC to build the headers and
libraries, and LLVM to build the compiler, linker, and utility tools.

## Prerequisites
Your host machine needs to have the following packages installed, which are
already part of the Shodan prerequisite pacakges

* CMake (>= 3.13.4)
* Ninja
* Clang

The source code of the toolchain is at
* [riscv-gnu-toolchain](https://github.com/riscv/riscv-gnu-toolchain): Checkout
the latest release tag, and checkout the submodule `riscv-binutils` with the
`rvv-1.0.x-zfh` branch.
* [llvm-project](https://github.com/llvm/llvm-project): Checkout the latest green
commit

## Build RISC-V Linux toolchain (64-bit)

### Build GCC:
```
$ mkdir -p <GCC_BUILD_PATH>
$ cd <GCC_BUILD_PATH>
$ <GCC_SRC_PATH>/configure \
  --srcdir=<GCC_SRC_PATH> \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --with-arch=rv64gc \
  --with-abi=lp64d \
  --with-cmodel=medany
$ make -C <GCC_BUILD_PATH> linux
```
Notice Linux requires the full general CPU extension support, i.e., rv64imafdc,
and the ABI also needs to support hard double-float modules. For 32-bit Linux,
build the toolchain with the flags of `--with-arch=rv32gc --with-abi=ilp32d`.

### Build LLVM:
```
$ cmake -B <LLVM_BUILD_PATH> \
		-DCMAKE_INSTALL_PREFIX=<TOOLCHAIN_OUT_DIR> \
		-DCMAKE_C_COMPILER=clang  -DCMAKE_CXX_COMPILER=clang++ \
		-DCMAKE_BUILD_TYPE=Release \
		-DLLVM_TARGETS_TO_BUILD="RISCV" \
		-DLLVM_ENABLE_PROJECTS="clang"  \
		-DLLVM_DEFAULT_TARGET_TRIPLE="riscv64-unknown-linux-gnu" \
		-DLLVM_INSTALL_TOOLCHAIN_ONLY=On \
		-DDEFAULT_SYSROOT=../sysroot \
		-G Ninja \
		<LLVM_SRC_PATH>/llvm
$ cmake --build  <LLVM_BUILD_PATH> --target install
```
For 32-bit, change the LLVM target triple to `riscv32-unknown-linux-gnu`.

## Build RISC-V bare-metal toolchain (32-bit)

### Build GCC:
```
$ mkdir -p <GCC_BUILD_PATH>
$ cd <GCC_BUILD_PATH>
$ <GCC_SRC_PATH>/configure \
  --srcdir=<GCC_SRC_PATH> \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --with-arch=rv32gc \
  --with-abi=ilp32 \
  --with-cmodel=medany
$ make -C <GCC_BUILD_PATH> newlib
```
Notice for bare-metal newlib there's no hard constraints on CPU feature and ABI
support. However, LLVM for bare-metal only supports soft-float modules, so the
GCC ABI setting needs to match that.

### Build LLVM:
```
$ cmake -B <LLVM_BUILD_PATH> \
		-DCMAKE_INSTALL_PREFIX=<TOOLCHAIN_OUT_DIR> \
		-DCMAKE_C_COMPILER=clang  -DCMAKE_CXX_COMPILER=clang++ \
		-DCMAKE_BUILD_TYPE=Release \
		-DLLVM_TARGETS_TO_BUILD="RISCV" \
		-DLLVM_ENABLE_PROJECTS="clang"  \
		-DLLVM_DEFAULT_TARGET_TRIPLE="riscv32-unknown-elf" \
		-DLLVM_INSTALL_TOOLCHAIN_ONLY=On \
		-DDEFAULT_SYSROOT=../riscv32-unknown-elf \
		-G Ninja \
		<LLVM_SRC_PATH>/llvm
$ cmake --build  <LLVM_BUILD_PATH> --target install
```
#### Build compiler-rt

This should not be necessary for the Shodan usage, but in case the compiler-rt
builtins is required in the project, it can be built with the additional commands

```
$ export PATH=<TOOLCHAIN_OUT_DIR>/bin:${PATH}
$ cmake -B <LLVM_BUILD_PATH>/compiler-rt
  -DCMAKE_INSTALL_PREFIX=$PREFIX \
  -DCMAKE_TRY_COMPILE_TARGET_TYPE=STATIC_LIBRARY \
  -DCMAKE_AR=<TOOLCHAIN_OUT_DIR>/bin/llvm-ar \
  -DCMAKE_NM=<TOOLCHAIN_OUT_DIR>/bin/llvm-nm \
  -DCMAKE_RANLIB=<TOOLCHAIN_OUT_DIR>/bin/llvm-ranlib \
  -DCMAKE_C_FLAGS="-march=rv32gc" \
  -DCMAKE_ASM_FLAGS="-march=rv32gc" \
  -DCMAKE_C_COMPILER=<TOOLCHAIN_OUT_DIR>/bin/clang \
  -DCMAKE_C_COMPILER_TARGET=riscv32-unknown-elf \
  -DCMAKE_ASM_COMPILER_TARGET=riscv32-unknown-elf \
  -DCOMPILER_RT_OS_DIR="clang/13.0.0/lib" \
  -DCMAKE_EXE_LINKER_FLAGS="-fuse-ld=lld" \
  -DCOMPILER_RT_BUILD_BUILTINS=ON \
  -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
  -DCOMPILER_RT_BUILD_XRAY=OFF \
  -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
  -DCOMPILER_RT_BUILD_MEMPROF=OFF \
  -DCOMPILER_RT_BUILD_PROFILE=OFF \
  -DCOMPILER_RT_BAREMETAL_BUILD=ON \
  -DCOMPILER_RT_DEFAULT_TARGET_ONLY=ON \
  -DLLVM_CONFIG_PATH=<LLVM_BUILD_PATH>/bin/llvm-config \
  -DCMAKE_C_FLAGS="-march=rv32gc -mno-relax" \
  -DCMAKE_ASM_FLAGS="-march=gv32gc -mno-relax" \
  -G "Ninja" <LLVM_SRC_PATH>/compiler-rt
$ cmake --build <LLVM_BUILD_PATH>/compiler-rt --target install
```

### Build newlib

The source code is at https://github.com/riscv/riscv-newlib

***NOTE: The GCC utility tools needs to be built first.***

```
$ mkdir -p <NEWLIB_BUILD_PATH>
$ cd <NEWLIB_BUILD_PATH>
$ <NEWLIB_SRC_PATH>/configure \
  --target=riscv32-unknown-elf \
  --prefix=<TOOLCHAIN_OUT_DIR> \
  --enable-newlib-io-long-double \
  --enable-newlib-io-long-long \
  --enable-newlib-io-c99-formats \
  --enable-newlib-register-fini \
  CC_FOR_TARGET=clang \
  CXX_FOR_TARGET=clang++ \
  CFLAGS_FOR_TARGET="-march=rv32gc -O2 -D_POSIX_MODE -mno-relax" \
  CXXFLAGS_FOR_TARGET="-march=rv32gc -O2 -D_POSIX_MODE -mno-relax"
$ make -j32
$ make install
```
## Test toolchain

Run
```
<TOOLCHAIN_OUT_DIR>/bin/<arch>-<os>-<abi>-gcc -v
```
to see the supported ABIs, architectures, library paths, etc.

Try to compile a simple c code (copied from CMake's package content, e.g.,
`/usr/share/cmake-3.18/Modules/CMakeTestCCompiler.cmake`)

```
#ifdef __cplusplus
# error "The CMAKE_C_COMPILER is set to a C++ compiler"
#endif
#if defined(__CLASSIC_C__)
int main(argc, argv)
  int argc;
  char* argv[];
#else
int main(int argc, char* argv[])
#endif
{ (void)argv; return argc-1;}
```
To build (32-bit bare-metal example)

```
$ <TOOLCHAIN_OUT_DIR>/bin/clang -c testCCompiler.c -O --target=riscv32
$ <TOOLCHAIN_OUT_DIR>/bin/riscv32-unknown-elf-gcc testCCompiler.o -o testCCompiler -march=rv32gc -mabi=ilp32
```
You can also use `readelf` to inspect the object file before building the binary
to see if the architecture and ABI match.