Vulkan is a new generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in a wide variety of devices from PCs and consoles to mobile phones and embedded platforms.
IREE includes a Vulkan/SPIR-V HAL backend designed for executing advanced ML models in a deeply pipelined and tightly integrated fashion on accelerators like GPUs.
This guide will walk you through using IREE's compiler and runtime Vulkan components. For generic Vulkan development environment set up and trouble shooting, please see this doc.
You should already have IREE cloned and building on your Linux machine. See the Getting Started on Linux with CMake or Getting Started on Linux with Bazel guide for instructions.
You may have a physical GPU with drivers supporting Vulkan. We also support using SwiftShader (a high performance CPU-based implementation of Vulkan).
Vulkan drivers implementing API version >= 1.2 are recommended. IREE requires the VK_KHR_timeline_semaphore extension (part of Vulkan 1.2), though it is able to emulate it, with performance costs, as necessary.
Please see Generic Vulkan Development Environment Setup and Troubleshooting for generic Vulkan concepts and development environment setup.
The dynamic_symbols_test checks if the Vulkan loader and a valid ICD are accessible.
Run the test:
# -- CMake -- $ export VK_LOADER_DEBUG=all $ cmake --build ../iree-build/ --target iree_hal_vulkan_dynamic_symbols_test $ ../iree-build/iree/hal/vulkan/iree_hal_vulkan_dynamic_symbols_test # -- Bazel -- $ bazel test iree/hal/vulkan:dynamic_symbols_test --test_env=VK_LOADER_DEBUG=all
Tests in IREE's HAL “Conformance Test Suite” (CTS) actually exercise the Vulkan HAL, which includes checking for supported layers and extensions.
Run the driver test:
# -- CMake -- $ export VK_LOADER_DEBUG=all $ cmake --build ../iree-build/ --target iree_hal_cts_driver_test $ ../iree-build/iree/hal/cts/iree_hal_cts_driver_test # -- Bazel -- $ bazel test iree/hal/cts:driver_test --test_env=VK_LOADER_DEBUG=all --test_output=all
If these tests pass, you can skip down to the next section.
If your system lacks a physical GPU with compatible Vulkan drivers, or you just want to use a software driver for predictable performance, you can set up SwiftShader's Vulkan ICD (Installable Client Driver).
IREE has a helper script for building SwiftShader from source using CMake:
$ bash build_tools/third_party/swiftshader/build_vk_swiftshader.sh
After building, the script will prompt your to add a variable VK_ICD_FILENAMES to your environment to tell the Vulkan loader to use the ICD. Assuming it was installed in the default location, this can be done via:
$ export VK_ICD_FILENAMES="${HOME?}/.swiftshader/Linux/vk_swiftshader_icd.json"
Bazel tests run in a sandbox, which environment variables may be forwarded to using the --test_env flag. A user.bazelrc file supporting each of the steps above looks like this (substitute for the {} paths):
test --test_env="LD_LIBRARY_PATH={PATH_TO_VULKAN_SDK}/x86_64/lib/"
test --test_env="LD_PRELOAD=libvulkan.so.1"
test --test_env="VK_ICD_FILENAMES={PATH_TO_IREE}/build-swiftshader/Linux/vk_swiftshader_icd.json"
Pass the flag -iree-hal-target-backends=vulkan-spirv to iree-translate:
# -- CMake -- $ cmake --build ../iree-build/ --target iree_tools_iree-translate $ ../iree-build/iree/tools/iree-translate -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv ./iree/tools/test/simple.mlir -o /tmp/module.vmfb # -- Bazel -- $ bazel run iree/tools:iree-translate -- -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv $PWD/iree/tools/test/simple.mlir -o /tmp/module.vmfb
Tip:
If successful, this may have no output. You can pass other flags like-print-ir-after-allto control the program.
Pass the flag -driver=vulkan to iree-run-module:
# -- CMake -- $ cmake --build ../iree-build/ --target iree_tools_iree-run-module $ ../iree-build/iree/tools/iree-run-module -module_file=/tmp/module.vmfb -driver=vulkan -entry_function=abs -function_inputs="i32=-2" # -- Bazel -- $ bazel run iree/tools:iree-run-module -- -module_file=/tmp/module.vmfb -driver=vulkan -entry_function=abs -function_inputs="i32=-2"
Install the Vulkan SDK, then run:
# -- CMake -- $ cmake --build ../iree-build/ --target iree_samples_vulkan_vulkan_inference_gui $ ../iree-build/iree/samples/vulkan/vulkan_inference_gui # -- Bazel -- $ bazel run iree/samples/vulkan:vulkan_inference_gui