| commit | a3e50b541cec6c6f4553747bac98b564c4ecf132 | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Wed Apr 15 08:40:53 2020 -0700 |
| committer | Copybara-Service <copybara-worker@google.com> | Wed Apr 15 08:41:55 2020 -0700 |
| tree | 6945b7dd8ddf96e6c943da3f383201a5dea5c142 | |
| parent | cc4455eea97c7ab0029e90aabdab7529bedc4238 [diff] |
Defining HAL target backend interfaces as part of #1168. This adds the new TargetBackend interface as sketched out in #1168. With this a bulk of the target-specific code for workgroup size/count and dispatch logic moves into this interface and support is extended for multiple target binaries. Serialization is now performed separately which makes the target backend code much easier to read and keeps the binary blobs out of the IR until the very end. There's still pending work here that will have to wait until upstream changes land for dynamic pass registration; right now the passes for each backend are run in a nested pass manager like the previous implementation however the design is setup such that we can yank out the iree-hal-translate-executables and iree-hal-serialize-executables passes when they are supported. Future changes will clean up the hal.executable.binary op to use a more free-form identifier and match that up with the dispatch recording. For now we only have one executable per target backend so things are fine, but as soon as we start generating variants for different SPIR-V versions/etc we'll have to adjust that. Closes https://github.com/google/iree/pull/1502 COPYBARA_INTEGRATE_REVIEW=https://github.com/google/iree/pull/1502 from google:benvanik-translate-interface 0a7a39ccd8748eaa41d0176ce74f4737b419f914 PiperOrigin-RevId: 306650298
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler that lowers ML models to a unified IR optimized for real-time mobile/edge inference against heterogeneous hardware accelerators. IREE also provides flexible deployment solutions for the compiled ML models.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logisitics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
For development, IREE supports both Bazel and CMake on Windows and Linux. We are working on enabling macOS support. For deployment, IREE aims to additionally cover Android and iOS.
Please see the Getting Started pages on IREE's documentation hub to configure, compile, and run IREE in your favorite development environment!
IREE hosts all its documentation and project status dashboards on GitHub Pages. We are still building up the website; please feel free to create issues for the documentation you'd like to see!
We also have some public talks that explain IREE's concepts and architecture:
IREE adopts a holistic approach towards ML model compilation: the IR produced contains both the scheduling logic, required to communicate data dependencies to low-level parallel pipelined hardware/API like Vulkan, and the execution logic, encoding dense computation on the hardware in the form of hardware/API-specific binaries like SPIR-V.
The architecture of IREE is best illustrated by the following picture:
Being compilation-based means IREE does not have a traditional runtime that dispatches “ops” to their fat kernel implemenations. What IREE provides is a toolbox for different deployment scenarios. It scales from running generated code on a particular API (such as emitting C code calling external DSP kernels), to a HAL (Hardware Abstraction Layer) that allows the same generated code to target multiple APIs (like Vulkan and Direct3D 12), to a full VM allowing runtime model loading for flexible deployment options and heterogeneous execution.
IREE aims to
IREE is still at its early stage; we have lots of exciting future plans. Please check out the long-term design roadmap and short-term focus areas.
We use GitHub Projects to track various IREE components and GitHub Milestones for major features and quarterly plans. Please check out for updated information.
| CI System | Build System | Platform | Status |
|---|---|---|---|
| GitHub Actions | Bazel | Linux | Workflow History |
| Kokoro | Bazel | Linux |  |
| Kokoro | CMake | Linux |  |
IREE is licensed under the terms of the Apache license. See LICENSE for more information.