| commit | 23f2828569d8bbe1d326a6a606090236f355885f | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Fri Feb 23 11:14:25 2024 -0800 |
| committer | GitHub <noreply@github.com> | Fri Feb 23 19:14:25 2024 +0000 |
| tree | d777da2de0e967a3d2bbfcc96c7a063888caf85d | |
| parent | 4a613b6f4ce89850268190f9e65582d9f112c632 [diff] |
Adding iree-benchmark-executable tool. (#16550) This allows for executables extracted from vmfb ZIP files, dumped by `--iree-hal-dump-executable-binaries-to=`, or compiled using `iree-compile --compile-mode=hal-executable` to be executed standalone without any compiled host code. Because it has no host code everything that the compiler was producing to dispatch must be provided as flags. Other features the runtime normal provides like fat binary/format selection are also not available and users must provide the correct executable format and file for their target. This is not intended to be used by normal humans - the easy path to benchmarking is to generate the benchmark executables via `--iree-hal-dump-executable-benchmarks-to=` or to author a benchmark in the frontend. `--help` shows some information and the included test shows VMVX. The `--binding=` flag used to provide input/output bindings matches the format of `--input=` in other tooling and can be used to provide zeroed sized buffers or parse or read contents from files. Simple example using the checked-in x86-64 test ELF: ``` iree-benchmark-executable \ --device=local-sync \ --executable_format=embedded-elf-x86_64 \ --executable_file=runtime/src/iree/hal/local/elf/testdata/elementwise_mul_x86_64.so \ --entry_point=0 \ --binding=4xf32=1,2,3,4 \ --binding=4xf32=100,200,300,400 \ --binding=4xf32 \ --workgroup_count=1,1,1 ``` (one benchmark will be run for each workgroup_count specified) This currently makes some assumptions that will not hold in the future, such as all bindings being in set 0 and densely packed. Future changes will probably make the binding flag specify set/binding ordinals. Currently command buffers are recorded within the dispatch inner loop to enable legacy backends (really just ROCM) to work - once ROCM is dead we can move the command buffer out of the loop and reuse the command buffer such that the only thing we do while timing is submit-and-wait. We can still have a mode for ALLOW_INLINE_EXECUTION for backends that can use it but the compiler will soon start generating secondary command buffers without that set so it may be removed from here.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.