Vulkan is a new generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in a wide variety of devices from PCs and consoles to mobile phones and embedded platforms.
IREE includes a Vulkan/SPIR-V HAL backend designed for executing advanced ML models in a deeply pipelined and tightly integrated fashion on accelerators like GPUs.
TODO(benvanik): specify required extensions and caps. Cross reference with gpuinfo.
The VK_KHR_timeline_semaphore extension is required. For systems where it is not natively available, the VK_LAYER_KHRONOS_timeline_semaphore compatibility layer may be used (see below).
TODO(benvanik): fill out hardware and OS matrix.
Vulkan applications interface with the Vulkan API through a series of systems including the Vulkan Loader which dispatches Vulkan functions to various layers and Installable Client Drivers (ICDs).
See also:
The Vulkan loader should already be installed on Vulkan-capable systems, but you might need to update it if the version that is installed is too old to use some of the more recently added Vulkan features.
To get a more recent version of the loader, these are some options:
If you build the loader from source, you may also need to set LD_PRELOAD=/path/to/libvulkan.so.1 (or equivalent) to bypass the installed loader.
On systems with a GPU and modern drivers, the Vulkan loader should already be present and configured to find and use the GPU's stable Vulkan driver. As needed, you can configure the loader to use alternate ICDs like SwiftShader (a high performance CPU-based implementation of Vulkan) or beta drivers.
Setting the VK_ICD_FILENAMES environment variable will instruct the Vulkan loader to read an ICD manifest file at that location. See build_vk_swiftshader.sh for a working example of how to build and configure SwiftShader as an ICD for IREE to use.
Setting the environment variable VK_LOADER_DEBUG=all will enable verbose logging during the loader initialization. This is especially useful when trying to verify expected paths are being searched for layers or driver JSON manifests.
The simplest test for ensuring that The Vulkan loader and IREE are correctly configured together is //iree/hal/vulkan:dynamic_symbols_test.
The --vulkan_validation_layers=true flag can be used to enable the standard validation meta-layer (VK_LAYER_LUNARG_standard_validation) containing all the common layers. Always run with this enabled unless you are profiling/ benchmarking as it‘s the ASAN-/TSAN-/MSAN-equivalent for Vulkan and will validate the usage and lifetime of API objects. If you’re seeing weird things the first step is to check the logs for validation warnings/errors.
You may need to preload your system Vulkan driver and define the driver shared object path prior to loading an application. For example, on Linux with NVIDIA:
LD_PRELOAD=libGLX_nvidia.so.0 LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu/ \
./bazel-bin/my/app
Failing to preload may result in an error like this (when VK_LOADER_DEBUG=all is defined):
DEBUG: Searching for ICD drivers named libGLX_nvidia.so.0, using default dir ERROR: dlopen: cannot load any more object with static TLS
If you are running under an SSH session to Linux you may also need to set the display if your drivers require X11:
DISPLAY=:0
TODO(benvanik): document optional extensions (e.g. VK_EXT_debug_utils)
IREE uses Vulkan's timeline semaphores from the VK_KHR_timeline_semaphore extension, which is core in Vulkan 1.2. For older Vulkan installs without native support for this extension, the Vulkan-ExtensionLayer project provides support through the VK_LAYER_KHRONOS_timeline_semaphore layer.
Developers may build/install this layer on their own, or using the Bazel or CMake build targets at build_tools/third_party/vulkan_extensionlayer.
After building the layer, set the VK_LAYER_PATH environment variable to include the path to the associated manifest file (see the docs for details).
Bazel runs tests in a sandbox and environment variables must be passed through to the test runner. Consider putting environment setup in a user.bazelrc. For example:
test --test_env="LD_LIBRARY_PATH={ABSOLUTE_PATH_TO_VULKAN_SDK}/x86_64/lib/"
test --test_env="LD_PRELOAD=libvulkan.so.1"
test --test_env="VK_LAYER_PATH=$VK_LAYER_PATH:{ABSOLUTE_PATH_TO_BUILD_DIR}/vulkan_extensionlayer/layers"
TODO(benvanik): minimum extensions and caps.
TODO(benvanik): subgroups in 1.1 https://www.khronos.org/blog/vulkan-subgroup-tutorial - basic/vote are supported: https://vulkan.gpuinfo.org/displayreport.php?id=6436#device
RenderDoc captures of IREE Vulkan execution may be recorded through RenderDoc‘s GUI and through IREE’s headless (command line) programs.
For IREE's GUI programs like samples/vulkan/vulkan_inference_gui, you should be able to launch the program through RenderDoc itself and control captures using the UI overlay it injects.
For IREE‘s headless programs, set the vulkan_enable_renderdoc flag to tell IREE to load RenderDoc, connect to it’s in-application API, and trigger capturing on its own. For example, this command runs iree-run-mlir on a simple MLIR file with some sample input values and saves a RenderDoc capture to the default location on your system (e.g. /tmp/RenderDoc/):
$ bazel build iree/tools:iree-run-mlir && LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$RENDERDOC_LIB bazel-bin/iree/tools/iree-run-mlir $PWD/iree/samples/vulkan/simple_mul.mlir -iree-hal-target-backends=vulkan-spirv -input-value="4xf32=1,2,3,4" -input-value="4xf32=2,4,6,8" -run-arg="--vulkan_renderdoc"
You can also launch IREE's headless programs through RenderDoc itself, just be sure to set the command line arguments appropriately. Saving capture settings in RenderDoc can help if you find yourself doing this frequently.
Note: RenderDoc version 1.7 or higher is needed to record captures from IREE's headless compute programs.