Vulkan is a new generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in a wide variety of devices from PCs and consoles to mobile phones and embedded platforms.
IREE includes a Vulkan/SPIR-V HAL backend designed for executing advanced ML models in a deeply pipelined and tightly integrated fashion on accelerators like GPUs.
This guide will walk you through using IREE's compiler and runtime Vulkan components.
You should already have IREE cloned and building on your Linux machine. See the Getting Started on Linux with CMake or Getting Started on Linux with Bazel guide for instructions.
You may have a physical GPU with drivers supporting Vulkan. We also support using SwiftShader (a high performance CPU-based implementation of Vulkan).
Vulkan API version > 1.2 is recommended where available. The VK_KHR_timeline_semaphore extension (part of Vulkan 1.2) is required, and the Vulkan-ExtensionLayer project can be used to enable it for drivers (like SwiftShader) without native support.
Vulkan applications interface with Vulkan “drivers”, “layers”, and “extensions” through the Vulkan loader. See LunarG's Architecture of the Vulkan Loader Interfaces page for more information.
The dynamic_symbols_test checks if the Vulkan loader and a valid ICD are accessible.
Run the test:
# -- CMake -- $ export VK_LOADER_DEBUG=all $ cmake --build build/ --target iree_hal_vulkan_dynamic_symbols_test $ ./build/iree/hal/vulkan/iree_hal_vulkan_dynamic_symbols_test # -- Bazel -- $ bazel test iree/hal/vulkan:dynamic_symbols_test --test_env=VK_LOADER_DEBUG=all
Tests in IREE's HAL “Conformance Test Suite” (CTS) actually exercise the Vulkan HAL, which includes checking for supported layers and extensions.
Run the device creation test:
# -- CMake -- $ export VK_LOADER_DEBUG=all $ cmake --build build/ --target iree_hal_cts_device_creation_test $ ./build/iree/hal/cts/iree_hal_cts_device_creation_test # -- Bazel -- $ bazel test iree/hal/cts:device_creation_test --test_env=VK_LOADER_DEBUG=all --test_output=all
If these tests pass, you can skip down to the next section.
If you see failures to find libvulkan.so.1 (the Vulkan loader), install it by either:
You may also need to set LD_LIBRARY_PATH and LD_PRELOAD to load the desired version of the loader. For example:
$ LD_LIBRARY_PATH={PATH_TO_VULKAN_SDK}/x86_64/lib/ $ LD_PRELOAD=libvulkan.so.1
If your system lacks a physical GPU with compatible Vulkan drivers, or you just want to use a software driver for predictable performance, you can set up SwiftShader's Vulkan ICD (Installable Client Driver).
IREE has a helper script for building SwiftShader from source using CMake:
$ bash build_tools/third_party/swiftshader/build_vk_swiftshader.sh
After building, set the VK_ICD_FILENAMES environment variable so the Vulkan loader uses the ICD:
$ export VK_ICD_FILENAMES=$PWD/build-swiftshader/Linux/vk_swiftshader_icd.json
If you are missing support for VK_KHR_timeline_semaphore, setup the extension layer.
Build:
# -- CMake -- $ cmake --build build/ --target vk_layer_khronos_timeline_semaphore # -- Bazel -- $ bazel build @vulkan_extensionlayer//:libVkLayer_khronos_timeline_semaphore.so @vulkan_extensionlayer//:VkLayer_khronos_timeline_semaphore_json
You should then also set the VK_LAYER_PATH environment variable to include the path to the built layer:
# -- CMake -- $ export VK_LAYER_PATH=$PWD/build/third_party/vulkan_extensionlayer/layers/:$VK_LAYER_PATH # -- Bazel -- $ export VK_LAYER_PATH=$PWD/bazel-bin/external/vulkan_extensionlayer/:$VK_LAYER_PATH
Bazel tests run in a sandbox, which environment variables may be forwarded to using the --test_env flag. A user.bazelrc file supporting each of the steps above looks like this (substitute for the {} paths):
test --test_env="LD_LIBRARY_PATH={PATH_TO_VULKAN_SDK}/x86_64/lib/"
test --test_env="LD_PRELOAD=libvulkan.so.1"
test --test_env="VK_ICD_FILENAMES={PATH_TO_IREE}/build-swiftshader/Linux/vk_swiftshader_icd.json"
test --test_env="VK_LAYER_PATH=$VK_LAYER_PATH:{PATH_TO_IREE}/bazel-bin/external/vulkan_extensionlayer/"
Pass the flag -iree-hal-target-backends=vulkan-spirv to iree-translate:
# -- CMake -- $ cmake --build build/ --target iree_tools_iree-translate $ ./build/iree/tools/iree-translate -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv ./iree/tools/test/simple.mlir -o /tmp/module.fb # -- Bazel -- $ bazel run iree/tools:iree-translate -- -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv $PWD/iree/tools/test/simple.mlir -o /tmp/module.fb
Tip:
If successful, this may have no output. You can pass other flags like-print-ir-after-allto control the program.
Pass the flag -driver=vulkan to iree-run-module:
# -- CMake -- $ cmake --build build/ --target iree_tools_iree-run-module $ ./build/iree/tools/iree-run-module -input_file=/tmp/module.fb -driver=vulkan -entry_function=abs -inputs="i32=-2" # -- Bazel -- $ bazel run iree/tools:iree-run-module -- -input_file=/tmp/module.fb -driver=vulkan -entry_function=abs -inputs="i32=-2"
Install the Vulkan SDK, then run:
# -- CMake -- $ cmake --build build/ --target iree_samples_vulkan_vulkan_inference_gui $ ./build/iree/samples/vulkan/vulkan_inference_gui # -- Bazel -- $ bazel run iree/samples/vulkan:vulkan_inference_gui
More documentation coming soon...