Google Git
Sign in
opensecura/3p/openxla/iree/40a49b82d07b6fad701dec845b0ab10cee98660a/./runtime/src/iree/hal/drivers/amdgpu
tree: d5f9d2be96f1096382b9ad48d0e4a77bfe0f0a8d
  1. cts/
  2. device/
  3. registration/
  4. util/
  5. allocator.c
  6. allocator.h
  7. api.h
  8. buffer.c
  9. buffer.h
  10. buffer_pool.c
  11. buffer_pool.h
  12. buffer_pool_test.cc
  13. BUILD.bazel
  14. channel.c
  15. channel.h
  16. CMakeLists.txt
  17. command_buffer.c
  18. command_buffer.h
  19. device_queue.c
  20. device_queue.h
  21. driver.c
  22. driver.h
  23. event.c
  24. event.h
  25. executable.c
  26. executable.h
  27. executable_cache.c
  28. executable_cache.h
  29. host_queue.c
  30. host_queue.h
  31. host_service.c
  32. host_service.h
  33. host_service_test.cc
  34. logical_device.c
  35. logical_device.h
  36. physical_device.c
  37. physical_device.h
  38. README.md
  39. semaphore.c
  40. semaphore.h
  41. semaphore_pool.c
  42. semaphore_pool.h
  43. semaphore_pool_test.cc
  44. system.c
  45. system.h
  46. system_test.cc
  47. virtual_queue.c
  48. virtual_queue.h
runtime/src/iree/hal/drivers/amdgpu/README.md

AMD GPU HAL Driver (amdgpu)

NOTE: the code is the authoritative documentation source. This document is an overview of the implementation and should be treated as informational only. See the linked files for details.

Quick Start

CMake

Configure CMake with the following options:

-DIREE_BUILD_COMPILER=ON
-DIREE_TARGET_BACKEND_ROCM=ON
-DIREE_HAL_DRIVER_AMDGPU=ON
-DIREE_HAL_AMDGPU_DEVICE_LIBRARY_TARGETS=gfx1100
-DIREE_ROCM_TEST_TARGET_CHIP=gfx1100

Bazel

The ROCM chip target defaults to gfx1100. Override for your hardware:

iree-bazel-test --//build_tools/bazel:rocm_test_target=gfx942 //runtime/src/iree/hal/drivers/amdgpu/cts/...

Substitute the architecture with your own. See therock_amdgpu_targets.cmake for a list of common targets. Future changes will include family support matching that file.

Use amdgpu to specify devices at runtime:

# Single logical device with all available physical devices:
iree-run-module --device=amdgpu
# Device ordinal 0 (danger, this may change across reboots):
iree-run-module --device=amdgpu:0
# Device with a stable UUID for a device:
iree-run-module --device=amdgpu://GPU-0e12865a3bf5b7ab
# Single logical device with the two devices given by their UUIDs:
iree-run-module --device=amdgpu://GPU-0e12865a3bf5b7ab,GPU-89e8bdf59a10cf6d
# Single logical device with physical devices with ordinals 2 and 3:
ROCR_VISIBLE_DEVICES=2,3 iree-run-module --device=amdgpu
# Two logical devices with two physical devices each:
iree-run-module --device=amdgpu://0,1 --device=amdgpu://2,3

Use amdgpu to specify the AMDGPU target when compiling programs:

iree-compile --iree-hal-target-device=amdgpu ...

Build Notes

HSA/ROCR Dependency

We maintain a fork of the HSA headers required for compilation as third_party/hsa-runtime-headers/. This fork may also contain tweaks not yet upstreamed required to use the headers in our build.

We require that at runtime a dynamic library with the name libhsa-runtime64.so exists on the path. This can be overridden programmatically when constructing the driver, via the --amdgpu_libhsa_search_path= flag if using the command line tools, via the IREE_HAL_AMDGPU_LIBHSA_PATH environment variable, or by just adding a directory containing the file to PATH.

It's recommended that developers check out a copy of the ROCR-Runtime and build it locally in whatever configuration they are using (debug/release/ASAN/etc). This allows for easier debugging and profiling as symbols are present and may be required to get recent features not available in platform installs. Eventually IREE will ship its own copy of the library (directly or indirectly) as part of the install packages such that only a relatively recent AMDGPU driver is required.

See HSA/ROCR Library for more information on our usage.

Device Library Compilation

Required CMake Options: -DIREE_BUILD_COMPILER=ON -DIREE_TARGET_BACKEND_ROCM=ON

Top-level Build Target: iree_hal_drivers_amdgpu_device_binaries

Currently IREE's CMake configuration must have the compiler enabled in order to build the runtime including the AMDGPU HAL implementation. This will be made better in the future (allowing for just building what we need instead of the full MLIR stack, using an existing ROCM install, etc). See Device Library for more information.

The device library should be compiled automatically when building the AMDGPU HAL driver and gets embedded inside the runtime binary so that no additional files are required at runtime.

The IREE_HAL_AMDGPU_DEVICE_LIBRARY_TARGETS CMake variable can be set to a list of target architectures to build the library for and bundle into the AMDGPU HAL library. Architectures not built into the library will fail to instantiate the driver at runtime.