Vulkan is a new generation graphics and compute API that provides high-efficiency, cross-platform access to modern GPUs used in a wide variety of devices from PCs and consoles to mobile phones and embedded platforms.
IREE includes a Vulkan/SPIR-V HAL backend designed for executing advanced ML models in a deeply pipelined and tightly integrated fashion on accelerators like GPUs.
This guide will walk you through using IREE's compiler and runtime Vulkan components. For generic Vulkan development environment set up and trouble shooting, please see this doc.
You should already have IREE cloned and building on your Windows machine. See the Getting Started on Windows with CMake or Getting Started on Windows with Bazel guide for instructions.
You must have a physical GPU with drivers supporting Vulkan. We support using SwiftShader (a high performance CPU-based implementation of Vulkan).
Vulkan drivers implementing API version >= 1.2 are recommended. IREE requires the VK_KHR_timeline_semaphore extension (part of Vulkan 1.2), though it is able to emulate it, with performance costs, as necessary.
Please see Generic Vulkan Development Environment Setup and Troubleshooting for generic Vulkan concepts and development environment setup.
The dynamic_symbols_test checks if the Vulkan loader and a valid ICD are accessible.
Run the test:
# -- CMake -- > set VK_LOADER_DEBUG=all > cmake --build build\ --target iree_hal_vulkan_dynamic_symbols_test > .\build\iree\hal\vulkan\iree_hal_vulkan_dynamic_symbols_test.exe # -- Bazel -- > bazel test iree/hal/vulkan:dynamic_symbols_test --test_env=VK_LOADER_DEBUG=all
Tests in IREE's HAL “Conformance Test Suite” (CTS) actually exercise the Vulkan HAL, which includes checking for supported layers and extensions.
Run the driver test:
# -- CMake -- > set VK_LOADER_DEBUG=all > cmake --build build\ --target iree_hal_cts_driver_test > .\build\iree\hal\cts\iree_hal_cts_driver_test.exe # -- Bazel -- > bazel test iree/hal/cts:driver_test --test_env=VK_LOADER_DEBUG=all --test_output=all
If these tests pass, you can skip down to the next section.
If your system lacks a physical GPU with compatible Vulkan drivers, or you just want to use a software driver for predictable performance, you can set up SwiftShader's Vulkan ICD (Installable Client Driver).
IREE has a helper script for building SwiftShader from source using CMake:
$ bash build_tools/third_party/swiftshader/build_vk_swiftshader.sh
After building, set the VK_ICD_FILENAMES environment variable so the Vulkan loader uses the ICD:
> $env:VK_ICD_FILENAMES = Resolve-Path "build-swiftshader/Windows/vk_swiftshader_icd.json"
Bazel tests run in a sandbox, which environment variables may be forwarded to using the --test_env flag. A user.bazelrc file using SwiftShader looks like this (substitute for the {} paths):
test --test_env="VK_ICD_FILENAMES={PATH_TO_IREE}\\build-swiftshader\\Windows\\vk_swiftshader_icd.json"
Pass the flag -iree-hal-target-backends=vulkan-spirv to iree-translate.exe:
# -- CMake -- > cmake --build build\ --target iree_tools_iree-translate > .\build\iree\tools\iree-translate.exe -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv .\iree\tools\test\simple.mlir -o .\build\module.fb # -- Bazel -- > bazel run iree/tools:iree-translate -- -iree-mlir-to-vm-bytecode-module -iree-hal-target-backends=vulkan-spirv .\iree\tools\test\simple.mlir -o .\build\module.fb
Tip:
If successful, this may have no output. You can pass other flags like-print-ir-after-allto control the program.
Pass the flag -driver=vulkan to iree-run-module.exe:
# -- CMake -- > cmake --build build\ --target iree_tools_iree-run-module > .\build\iree\tools\iree-run-module.exe -input_file=.\build\module.fb -driver=vulkan -entry_function=abs -inputs="i32=-2" # -- Bazel -- > bazel run iree/tools:iree-run-module -- -input_file=.\build\module.fb -driver=vulkan -entry_function=abs -inputs="i32=-2"
Install the Vulkan SDK, then run:
# -- CMake -- > cmake --build build\ --target iree_samples_vulkan_vulkan_inference_gui > .\build\iree\samples\vulkan\vulkan_inference_gui.exe # -- Bazel -- > bazel run iree/samples/vulkan:vulkan_inference_gui