Generate and Inspect IREE Executables

In this doc, we explain the process of generating the IREE executables and inspecting the excutables with various tools.

Generate IREE executables

IREE's main codegen tools are iree-opt (generate MLIR representations) and iree-translate (generate the IREE bytecode modules). IREE codegen flow is based on LLVM and MLIR, so it utilizes the typical LLVM flags to define the machine targets. For example, to generate the IREE bytecode module from a vector multiply MLIR:

${OUT}/host/iree_compiler/install/bin/iree-translate \
    -iree-input-type=mhlo \
    -iree-mlir-to-vm-bytecode-module \
    -iree-hal-target-backends=dylib-llvm-aot \
    -iree-llvm-target-triple=riscv32-pc-linux-elf \
    -iree-llvm-target-cpu=generic-rv32 \
    -iree-llvm-target-cpu-features="+m,+f" \
    -iree-llvm-target-abi=ilp32 \
    -iree-llvm-embedded-linker-path=${OUT}/host/iree_compiler/install/bin/lld \
    ${ROOTDIR}/toolchain/iree/iree/samples/simple_embedding/simple_embedding_test.mlir \
    -o /tmp/simple_mul-llvm_aot.vmfb

With the options of:

  • iree-hal-target-backends: The HAL device (library + dispatcher) target for the workload. In Shodan, the supported targets are:
    • dylib-llvm-aot: the dynamic library for LLVM ahead-of-time (AOT) compilation.
    • vmvx
  • iree-llvm-target-triple: The flag is populated to LLVM target triple.
  • iree-llvm-target-cpu: The flag populated to LLVM target cpu. It can be pre-defined cpu targets or the generic ones (link).
  • iree-llvm-target-cpu-features: The flag is populated to LLVM target features for extra CPU extensions. For RISC-V, it can include the typical ISA extensions, such as multiplication/division, atomic, floating point, and compression ISA support. For the vector extension, it can be enabled with “+experimental-v”.
  • iree-llvm-target-abi: The flag is polulated to LLVM target abi.
  • iree-llvm-embedded-linker-path: Linker for the device executable. It is recommanded to use the lld from the IREE compiler release.

Some extra options:

  • iree-llvm-keep-linker-artifacts: Aside from the generated bytecode flatbuffer (with schema), the intermediate linker is generated at /tmp/<module_name>_linked_<hal_target>-xxxxxx.so. (the exact path is printed in the stdout).
  • iree-vm-emit-polyglot-zip: Instead of the previous flag, by enabling this flag you can extract the .so from the .vmfb file with 7z
7z e -aoa -bb0 <vmfb> -y
  • riscv-v-vector-bits-min and riscv-v-fixed-length-vector-lmul-max: If the vector extension is enabled in iree-llvm-target-cpu-features, the RVV VLS (vector length specific) code will be generated with the vector length specified by these two options.
  • iree-llvm-debug-symbols: Add the debug information in the executable. Setting this to false at the production executable can reduce the workload size.

Inspect IREE executables

To render the bytecode flatbuffer in a text file, the iree-dump-module tool can print out the content based on the descriptors in the schema, including the weights in the ML model.

${OUT}/host/iree-compiler/install/bin/iree-dump-module <vmfb file>

With the linker artifact enabled, The .so file contends the dispatch functions of the workload, and the user can use the objdump in the RISC-V toolchain to retrieve the assembler content. For example, to render the executable (.text) segment

${CACHE}/toolchain_iree_rv32imf/bin/riscv32-unknown-elf-objdump -d <linker artifact>

Note:

  • If you use llvm-objdump to retrieve the assembler content, the RVV ISAs could be rendered as unknown.

Inspect IREE IR lowering

Use iree-opt to check the IR output

${OUT}/host/iree-compiler/install/bin/iree-opt \
  -print-ir-after-all \
  -iree-transformation-pipeline \
  -iree-hal-target-backends=dylib-llvm-aot \
  -iree-llvm-target-triple=riscv32-pc-linux-elf \
  -iree-llvm-target-cpu=generic-rv32 \
  -iree-llvm-target-cpu-features="+m,+f" \
  -iree-llvm-target-abi=ilp32 \
  <MLIR input>