amdgpu)NOTE: the code is the authoritative documentation source. This document is an overview of the implementation and should be treated as informational only. See the linked files for details.
Configure CMake with the following options:
-DIREE_BUILD_COMPILER=ON -DIREE_TARGET_BACKEND_ROCM=ON -DIREE_HAL_DRIVER_AMDGPU=ON -DIREE_HAL_AMDGPU_DEVICE_LIBRARY_TARGETS=all -DIREE_ROCM_TEST_TARGET_CHIP=gfx1100
Build tools with the AMDGPU runtime driver registered and with device artifacts compiled for your local GPU architecture:
iree-bazel-build //tools:iree-compile //tools:iree-run-module \ --iree_drivers=amdgpu,cuda,hip,local-sync,local-task,vulkan \ --//build_tools/bazel:rocm_test_target=gfx1100
The ROCM chip target defaults to gfx1100. Override for your hardware:
iree-bazel-test --//build_tools/bazel:rocm_test_target=gfx942 //runtime/src/iree/hal/drivers/amdgpu/cts/...
Substitute the architecture with your own. See therock_amdgpu_targets.cmake for the target and generic family vocabulary mirrored by the embedded device library build.
Use amdgpu to specify devices at runtime:
# Single logical device with all available physical devices: iree-run-module --device=amdgpu # Device ordinal 0 (danger, this may change across reboots): iree-run-module --device=amdgpu:0 # Device with a stable UUID for a device: iree-run-module --device=amdgpu://GPU-0e12865a3bf5b7ab # Single logical device with the two devices given by their UUIDs: iree-run-module --device=amdgpu://GPU-0e12865a3bf5b7ab,GPU-89e8bdf59a10cf6d # Single logical device with physical devices with ordinals 2 and 3: ROCR_VISIBLE_DEVICES=2,3 iree-run-module --device=amdgpu # Two logical devices with two physical devices each: iree-run-module --device=amdgpu://0,1 --device=amdgpu://2,3
Use amdgpu to specify the AMDGPU target when compiling programs:
iree-compile --iree-hal-target-device=amdgpu ...
For a direct Bazel-built smoke test, compile with the AMDGPU target device and run the resulting VMFB with the AMDGPU HAL driver:
bazel-bin/tools/iree-compile \ --iree-input-type=stablehlo \ --iree-hal-target-device=amdgpu \ --iree-rocm-target=gfx1100 \ --iree-rocm-bc-dir=bazel-bin/external/_main~iree_extension~amdgpu_device_libs/bitcode \ tests/e2e/stablehlo_models/mnist_fake_weights.mlir \ -o=/tmp/mnist_fake_amdgpu.vmfb bazel-bin/tools/iree-run-module \ --device=amdgpu \ --module=/tmp/mnist_fake_amdgpu.vmfb \ --function=predict \ --input=1x28x28x1xf32 \ --expected_output='1x10xf32=0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1'
Prefer this explicit two-step flow when debugging AMDGPU target-device behavior. iree-run-mlir --device=amdgpu currently relies on generic device-to-compiler flag inference and may select the legacy ROCm/HIP target path instead of the AMDGPU HAL target.
To capture a Tracy trace of the same runtime path, use the Bazel trace wrapper. The wrapper requires a tracy-capture binary in PATH, or one supplied through IREE_TRACY_CAPTURE:
IREE_TRACY_CAPTURE=/path/to/tracy-capture \ build_tools/bin/iree-bazel-run \ --trace \ --trace_name=fake_mnist \ //tools:iree-run-module \ --iree_drivers=amdgpu,cuda,hip,local-sync,local-task,vulkan \ --//build_tools/bazel:rocm_test_target=gfx1100 \ -- \ --device=amdgpu \ --module=/tmp/mnist_fake_amdgpu.vmfb \ --function=predict \ --input=1x28x28x1xf32 \ --expected_output='1x10xf32=0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1 0.1'
The AMDGPU HAL driver is a native HSA/ROCR backend. It does not route through HIP or the legacy ROCm HAL driver. The major runtime objects are:
Normal execution only depends on the HSA runtime and the embedded device library. Optional profiling modes dynamically load ROCm profiling libraries only when the selected mode needs them.
The AMDGPU driver is one of the primary producers for IREE's HAL-native profiling and replay tools:
--device_profiling_mode=queue-events,device-queue-events,dispatch-events captures queue submissions, device-side queue timing, and per-dispatch timing into a .ireeprof bundle.--device_profiling_mode=executable-metadata,counters adds executable export metadata and selected hardware/software counters.--device_profiling_mode=executable-traces captures heavy ATT/SQTT artifacts for filtered dispatches.--device_replay_output=/tmp/model.ireereplay records a HAL-level replay stream that can be run, benchmarked, and profiled independently of the original application.Useful inspection commands:
iree-profile summary /tmp/model.ireeprof iree-profile dispatch --format=jsonl /tmp/model.ireeprof iree-profile export --format=ireeperf-jsonl \ --output=/tmp/model.ireeperf.jsonl /tmp/model.ireeprof uvx --with perfetto --with protobuf python "$(command -v iree-profile-render)" \ --format=perfetto \ /tmp/model.ireeperf.jsonl -o /tmp/model.pftrace iree-profile att --rocm_library_path=/opt/rocm/lib /tmp/model-att.ireeprof
See the website documentation for the full workflows:
We maintain a fork of the HSA headers required for compilation as third_party/hsa-runtime-headers/. This fork may also contain tweaks not yet upstreamed required to use the headers in our build.
We require that at runtime a dynamic library with the name libhsa-runtime64.so exists on the path. This can be overridden programmatically when constructing the driver, via the --amdgpu_libhsa_search_path= flag if using the command line tools, via the IREE_HAL_AMDGPU_LIBHSA_PATH environment variable, or by just adding a directory containing the file to PATH.
It's recommended that developers check out a copy of the ROCR-Runtime and build it locally in whatever configuration they are using (debug/release/ASAN/etc). This allows for easier debugging and profiling as symbols are present and may be required to get recent features not available in platform installs. Eventually IREE will ship its own copy of the library (directly or indirectly) as part of the install packages such that only a relatively recent AMDGPU driver is required.
See HSA/ROCR Library for more information on our usage.
Counter and ATT/SQTT capture use ROCm's aqlprofile library through a small dynamic-loader shim. Normal execution, queue timing, dispatch timing, replay, statistics, and Perfetto export do not require this library.
When counters or executable-traces profiling is requested, the driver looks for an aqlprofile-compatible library in this order:
IREE_HAL_AMDGPU_LIBAQLPROFILE_PATH;The iree-profile att decoder also needs ROCm decode libraries. Pass --rocm_library_path=/opt/rocm/lib, set IREE_HAL_AMDGPU_LIBAQLPROFILE_PATH, or rely on the platform search path.
Required CMake Options: -DIREE_BUILD_COMPILER=ON -DIREE_TARGET_BACKEND_ROCM=ON
Top-level Build Target: iree_hal_drivers_amdgpu_device_binaries
Currently IREE's CMake configuration must have the compiler enabled in order to build the runtime including the AMDGPU HAL implementation. This will be made better in the future (allowing for just building what we need instead of the full MLIR stack, using an existing ROCM install, etc). See Device Library for more information.
The device library should be compiled automatically when building the AMDGPU HAL driver and gets embedded inside the runtime binary so that no additional files are required at runtime.
The IREE_HAL_AMDGPU_DEVICE_LIBRARY_TARGETS CMake variable defaults to all, which embeds LLVM generic ISA code objects covering every currently known AMDGPU device library target. Packagers can set it to a smaller list of exact target architectures, LLVM generic ISA targets, TheRock-style generic target families, or TheRock-style product bundles. Exact targets use the HSA ISA spelling, such as gfx1100. LLVM generic ISA targets use spellings such as gfx11-generic. Generic families use the TheRock family spelling, such as gfx110X-all, and product bundles use spellings such as dgpu-all or igpu-all. These selectors expand to the smallest known compatible code object set instead of one code object per exact GPU. Architectures not built into the library will fail to instantiate the driver at runtime.
The Bazel build exposes the same selector vocabulary through //runtime/src/iree/hal/drivers/amdgpu/device/binaries:targets:
iree-bazel-build --//runtime/src/iree/hal/drivers/amdgpu/device/binaries:targets=igpu-all //runtime/src/iree/hal/drivers/amdgpu:amdgpu
See device/binaries/README.md for the target map update flow, the generated Bazel/CMake/runtime fragments, and the TheRock/LLVM sources that should be checked when adding support for a new architecture.