Google Git
Sign in
opensecura / sw / vec_iree / 96e565428220bc9cfa13f21ac374c8cef2554e94
commit96e565428220bc9cfa13f21ac374c8cef2554e94[log] [tgz]
authorLun Dong <lundong@google.com>Sat May 13 13:25:12 2023 -0700
committerLun Dong <lundong@google.com>Mon Dec 11 13:02:05 2023 -0800
treeee19c18a0e394fafb90c64fe7a086eb169f11db6
parent12974d4945757211be160e2ae254858c1172474b [diff]
vec_iree: Add support for LLVM ukernel via plugin importer

For Kelvin, we used efficient "aconv" op to implement custom mmt4d
ukernel and achieved substantial performance improvement for many models.

For Springbok, LLVM ukernel also works, but we disable it for
now as it currently does not perform better than current version with
RVV_ON.

We also did clean-ups on the folder "vmvx_ukernel" to accommodate latest
VMVX and LLVM ukernel APIs and design.

Some further notes:
- We renamed folder vmvx_ukernel to ukernel as it now supports for both VMVX and LLVM.
- We removed custom ukerenl/query_tile_sizes.c to use IREE default for VMVX ukernel, as it was arbitrarily chosen before and VMVX ukernel now uses dynamic tile sizes.
- We only enabled LLVM ukernel for Kelvin for now, as explained above.
- For our custom ukernel implementation in ukernel/mmt4d_tile.c, we only
  considered the case of using Kelvin intrincics or using RVV. Generic
  ukernel case was omitted as it's not our interest and is already available as IREE builtin default ukernel.

Change-Id: Ia96bcc6a676067cab412c8bc89788faa73403b6d
11 files changed
tree: ee19c18a0e394fafb90c64fe7a086eb169f11db6
  1. audio_prep/
  2. build_tools/
  3. cmake/
  4. device/
  5. model_util/
  6. native_log/
  7. risp4ml/
  8. samples/
  9. ssd_postprocess/
  10. ukernel/
  11. CMakeLists.txt
  12. CONTRIBUTING.md
  13. LICENSE
  14. README.md
  15. sparrow_config.h
README.md

Springbok Model Execution Repository

This project contains ML model codegen and execution libraries/examples. The artifacts built from this project is targeted for RISC-V 32-bit baremetal machine (Springbok or Kelvin).

Prerequisite

If you get this project from Project Shodan manifest, you are all set. If not, you need to have following projects as well to build the project successfully.

This project needs to be at <dir>/sw/vec_iree

Code structure

  • audio_prep: Audio preprocessing libarary
  • build_tools: Utility scripts for the project
  • cmake: CMake Macros for the project
  • device: Device HAL driver library
  • model_util: Runtime utility library for model execution
  • native_log: BSP support for the Linux userspace emulation
  • risp4ml: Vision preprocessing library (Reduced ISP for ML)
  • samples: Codegen and execution of ML models based on IREE
    • branch_mul: Branch multiplication example
    • microbenchmarks: Currently only contain a convolution 1x1 example
    • simple_vec_mul: Point-wise vector multiplication examples
  • ssd_postprocess: Vision postprocessing library for Single-Shot Detectors (SSD)
  • ukernel: Custom microkernel library for VMVX and LLVM

Build the project

If you are running from Project Shodan repo, you can build the artifacts with m iree. If you download this project alone, you need to download/build IREE's host compiler, then go to the top level directory.

For Springbok

  cmake -B <output dir> -G Ninja \
     -DCMAKE_TOOLCHAIN_FILE="$(realpath sw/vec_iree/cmake/riscv_iree.cmake)" \
     -DCMAKE_BUILD_TYPE=MinSizeRel \
     -DIREE_HOST_BIN_DIR="$(IREE_COMPILER_DIR)/install/bin" \
     -DRISCV_TOOLCHAIN_ROOT=$(TOOLCHAINRV32_PATH) \
     <dir>/sw/vec_iree
  cmake --build <output dir> --target all

For Kelvin

  cmake -B <output dir> -G Ninja \
     -DCMAKE_TOOLCHAIN_FILE="$(realpath sw/vec_iree/cmake/riscv_iree_gcc.cmake)" \
     -DCMAKE_BUILD_TYPE=MinSizeRel \
     -DIREE_HOST_BIN_DIR="$(IREE_COMPILER_DIR)/install/bin" \
     -DRISCV_TOOLCHAIN_ROOT=$(TOOLCHAINRV32_PATH) \
     <dir>/sw/vec_iree
  cmake --build <output dir> --target all

Run the executables

You can run RENODE emulation with the built executables. In Shodan repo, sim_springbok for Renode simulation on Springbok; sim_kelvin for Renode simulation on Kelvin. Out-of-repo emulation is possible but requires more steps.

Test the executables

In Shodan, this project utilizes LLVM lit and FileCheck to test the ML executable performance. The tests are defined in the *_test.run files under <output dir>. To run the test, at the Shodan top level directory

lit --path <Filecheck dir> -a <output dir>

Filecheck in Debian testing is under /usr/lib/llvm-14/bin

Default is renode tests for Springbok. Add-D RUNNER=kelvin to enable renode tests for Kelvin. Add-D RUNNER=kelvin_hw to enable renode tests for Kelvin HW.