| commit | ad23611856453012ac644aecc91a42c4df54192e | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Mon Dec 05 23:12:38 2022 -0800 |
| committer | GitHub <noreply@github.com> | Mon Dec 05 23:12:38 2022 -0800 |
| tree | 44d5bb4199bf2e9606a56933d38d5f6ca3a66695 | |
| parent | 27be42f696e5c6c98d31243cbcc8a169edb18a84 [diff] | |
| parent | 05f29e2cae7a6f06ab15ed94b195e5f4ff4277cf [diff] |
Adding plumbing and samples of custom CUDA/SPIR-V/CPU dispatch code. (#11287)
This adds a skeleton workflow for declaring external objects that are
able to be referenced by the compiler all the way from the high-level
dialects (flow, at least) and a sample demonstrating how device
functions in .cu, .glsl, and .o files can be connected end-to-end with
execution. The end goal is to allow codegen backends - including
extensions - to either select from precompiled object files or
generate/JIT their own as part of translation.
The core feature added is the `#hal.executable.object` attribute that
allows executable variants to have extra object files specified:
```mlir
hal.executable private @executable {
// Variant linking in an x86-64 object file containing external functions.
hal.executable.variant public @x86_64, target = #x86_64_target, objects = [
// Object files linked into the executable.
#hal.executable.object<{
// Referencing a file path on disk but could also have the data embedded.
path = "samples/custom_dispatch/cpu/embedded/functions.o"
}>
```
The attribute also allows for data to be directly embedded and (in the
future) specified via MLIR external resources. When using file paths
either the absolute path, a path relative to the current working
directory, or a relative path joined with search paths will be tried.
`--iree-hal-executable-object-search-path=` can be specified multiple
times to add to the search paths.
This can be extended as required for new backends (.metallibs and
archives) as well as various formats for the existing ones (linking .spv
files together, .bc instead of .o for LLVM IR, etc). There's some work
to be done around interactions with linking phases that may limit the
usefulness of these stages for cross-compilation but at least for now we
can rely on a bleeding-edge user wanting to hand-author these dispatch
functions not caring about it yet. Since objects are specified
per-variant it's possible for a user to still cross-compile and
multi-target just with a bit more IR (today). It's also possible to
specify the objects at the stream level and (if we have a good story for
tensor -> pointers) we could allow it at the flow/input-level too.
Some future ergonomics improvements are tracked in #11289.
In particular there will be a way to avoid the boilerplate the CPU side
requires by having us generate that automatically for some very basic
interfaces (limited access to workgroup params and the environment).IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.