| commit | 9955ff07845093788e09a4d6b9299d07d10cfb62 | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Wed Feb 15 22:23:02 2023 -0800 |
| committer | Ben Vanik <ben.vanik@gmail.com> | Fri Feb 17 09:04:37 2023 -0800 |
| tree | b981ad47bb000780b13e1c34c810d84eae0aebc9 | |
| parent | f560f42ea8ed66a64767546e03b6aec8834b6fd4 [diff] |
Adding `--iree-hal-substitute-executable-*=` flags. This allows for specifying one or more `executable_name=file.xxx` pairs that each replace a `hal.executable` op with the `executable_name` with the contents of `file.xxx`. .mlir/.mlirbc files are loaded and a `hal.executable` with the matching name is used as a replacement while any other file type will cause the original executable to be externalized and linked with the specified file (.ptx/.spv/etc). The additional `--iree-hal-substitute-executable-*-from=` flags allow for scanning a directory for executables by name to build the substitution mapping. Only files in the executable_name or module_executable_name form will be substituted but we can extend this in the future to support variant naming. Because of phase ordering constraints around where codegen is able to mutate host-related code such as workgroup count calculations there are two flag sets: `--iree-hal-substitute-executable-source=name=file.xxx` `--iree-hal-substitute-executable-sources-from=path/` and `--iree-hal-substitute-executable-object=name=file.xxx` `--iree-hal-substitute-executable-objects-from=path/` Sources are substituted immediately prior to benchmark generation and when substituting target objects (.ptx, .spv, etc) require that it's ok to skip codegen (workgroup count calculation is not dependent on root op detection, etc). Objects are substituted immediately after codegen for use in cases where codegen is to generate the host code. There are uses for both depending on what the input IR is and what the developer wants to modify (host code, device code, or both). The primary developer workflows this covers are: 1. dump executable sources via `--iree-hal-dump-executable-sources-to=` and modify them, potentially running any number of iree-opt passes, before linking them back in to the original program they came from 2. author custom implementations ala the custom_dispatch sample in target toolchains (.cu -> .ptx, .glsl -> .spv, etc) and use those in full programs without needing to modify the compiler 3. do either of the above and use the substituted executable for microbenchmarking via `--iree-hal-dump-executable-benchmarks-to=` (so one can easily microbenchmark handwritten kernels) Progress on #12222 (rest is orthogonal).
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.