Adding iree-benchmark-executable tool. (#16550)
This allows for executables extracted from vmfb ZIP files, dumped by
`--iree-hal-dump-executable-binaries-to=`, or compiled using
`iree-compile --compile-mode=hal-executable` to be executed standalone
without any compiled host code. Because it has no host code everything
that the compiler was producing to dispatch must be provided as flags.
Other features the runtime normal provides like fat binary/format
selection are also not available and users must provide the correct
executable format and file for their target.
This is not intended to be used by normal humans - the easy path to
benchmarking is to generate the benchmark executables via
`--iree-hal-dump-executable-benchmarks-to=` or to author a benchmark in
the frontend.
`--help` shows some information and the included test shows VMVX. The
`--binding=` flag used to provide input/output bindings matches the
format of `--input=` in other tooling and can be used to provide zeroed
sized buffers or parse or read contents from files.
Simple example using the checked-in x86-64 test ELF:
```
iree-benchmark-executable \
--device=local-sync \
--executable_format=embedded-elf-x86_64 \
--executable_file=runtime/src/iree/hal/local/elf/testdata/elementwise_mul_x86_64.so \
--entry_point=0 \
--binding=4xf32=1,2,3,4 \
--binding=4xf32=100,200,300,400 \
--binding=4xf32 \
--workgroup_count=1,1,1
```
(one benchmark will be run for each workgroup_count specified)
This currently makes some assumptions that will not hold in the future,
such as all bindings being in set 0 and densely packed. Future changes
will probably make the binding flag specify set/binding ordinals.
Currently command buffers are recorded within the dispatch inner loop to
enable legacy backends (really just ROCM) to work - once ROCM is dead we
can move the command buffer out of the loop and reuse the command buffer
such that the only thing we do while timing is submit-and-wait. We can
still have a mode for ALLOW_INLINE_EXECUTION for backends that can use
it but the compiler will soon start generating secondary command buffers
without that set so it may be removed from here.
diff --git a/.github/workflows/validate_and_publish_release.yml b/.github/workflows/validate_and_publish_release.yml
index 2f9b8a8..3e060fe 100644
--- a/.github/workflows/validate_and_publish_release.yml
+++ b/.github/workflows/validate_and_publish_release.yml
@@ -59,6 +59,9 @@
TRACY_NO_INVARIANT_CHECK=1 IREE_PY_RUNTIME=tracy \
python -m iree.runtime._package_test
# Binaries from the tarball
+ - name: Run iree-benchmark-executable
+ id: run_iree_benchmark_executable
+ run: ./bin/iree-benchmark-executable --help
- name: Run iree-benchmark-module
id: run_iree_benchmark_module
run: ./bin/iree-benchmark-module --help
@@ -90,6 +93,9 @@
- name: Py iree-run-module
id: py_iree-run-module
run: iree-run-module --help
+ - name: Py iree-benchmark-executable
+ id: py_iree_benchmark_executable
+ run: iree-benchmark-executable --help
- name: Py iree-benchmark-module
id: py_iree_benchmark_module
run: iree-benchmark-module --help
