Generate and Inspect IREE Executables

In this doc, we explain the process of generating the IREE executables and inspecting the excutables with various tools.

Generate IREE executables

IREE's main codegen tools are iree-opt (generate MLIR representations) and iree-translate (generate the IREE bytecode modules). IREE codegen flow is based on LLVM and MLIR, so it utilizes the typical LLVM flags to define the machine targets. For example, to generate the IREE bytecode module from a vector multiply MLIR:

${OUT}/host/iree-compiler/install/bin/iree-translate \
    -iree-input-type=mhlo \
    -iree-mlir-to-vm-bytecode-module \
    -iree-hal-target-backends=dylib-llvm-aot \
    -iree-llvm-target-triple=riscv32-pc-linux-elf \
    -iree-llvm-target-cpu=generic-rv32 \
    -iree-llvm-target-cpu-features="+m,+f" \
    -iree-llvm-target-abi=lp32d \
    -iree-llvm-link-embedded=true \
    ${ROOTDIR}/toolchain/iree/iree/samples/simple_embedding/simple_embedding_test.mlir \
    -o /tmp/simple_mul-llvm_aot.vmfb

With the options of:

  • iree-hal-target-backends: The HAL device (library + dispatcher) target for the workload. In Shodan, the supported targets are:
    • dylib-llvm-aot: the dynamic library for LLVM ahead-of-time (AOT) compilation.
    • vmvx
  • iree-llvm-target-triple: The flag is populated to LLVM target triple.
  • iree-llvm-target-cpu: The flag populated to LLVM target cpu. It can be pre-defined cpu targets or the generic ones (link).
  • iree-llvm-target-cpu-features: The flag is populated to LLVM target features for extra CPU extensions. For RISC-V, it can include the typical ISA extensions, such as multiplication/division, atomic, floating point, and compression ISA support. For the vector extension, it can be enabled with “+experimental-v”.
  • iree-llvm-target-abi: The flag is polulated to LLVM target abi.
  • iree-llvm-link-embedded: Generate the executable into a platform-agnostic ELF and use the ELF loader to load the library instead of using dlopen and dlsym. This creates the least overhead in the executable, and the simplest linker during codegen without dynamic library linkage; therefore, it will be the targeted path for Shodan. However, it is WIP and currently is not set as the default path.

Some extra options:

  • iree-llvm-keep-linker-artifacts: Aside from the generated bytecode flatbuffer (with schema), the intermediate linker is generated at /tmp/<module_name>_linked_<hal_target>-xxxxxx.so. (the exact path is printed in the stdout).
  • riscv-v-vector-bits-min and riscv-v-fixed-length-vector-lmul-max: If the vector extension is enabled in iree-llvm-target-cpu-features, the RVV VLS (vector length specific) code will be generated with the vector length specified by these two options.
  • iree-llvm-debug-symbols: Add the debug information in the executable. Setting this to false at the production executable can reduce the workload size.

Inspect IREE executables

To render the bytecode flatbuffer in a text file, the iree-dump-module tool can print out the content based on the descriptors in the schema, including the weights in the ML model.

${OUT}/host/iree-compiler/install/bin/iree-dump-module <vmfb file>

With the linker artifact enabled, The .so file contends the dispatch functions of the workload, and the user can use the objdump in the RISC-V toolchain to retrieve the assembler content. For example, to render the executable (.text) segment

${CACHE}/toolchain_iree_rv32imf/bin/riscv32-unknown-elf-objdump -d <linker artifact>

Note:

  • If you use llvm-objdump to retrieve the assembler content, the RVV ISAs could be rendered as unknown.

Inspect IREE IR lowering

Use iree-opt to check the IR output

${OUT}/host/iree-compiler/install/bin/iree-opt \
  -print-ir-after-all \
  -iree-transformation-pipeline \
  -iree-hal-target-backends=dylib-llvm-aot \
  -iree-llvm-target-triple=riscv32-pc-linux-elf \
  -iree-llvm-target-cpu=generic-rv32 \
  -iree-llvm-target-cpu-features="+m,+a,+f" \
  -iree-llvm-target-abi=lp32d \
  -iree-llvm-link-embedded=true \
  <MLIR input>