commit	69f19b3bacab0af1695b27f997eb9b07c8636df8	[log] [tgz]
author	Geoffrey Martin-Noble <gcmn@google.com>	Wed Dec 02 14:11:29 2020 -0800
committer	Geoffrey Martin-Noble <gcmn@google.com>	Wed Dec 02 14:11:59 2020 -0800
tree	ba757c940c6b9010637008d4064e3c64b3370eec
parent	a013c431207e3e8a68eba3ec3a596a4a7368d75c [diff]
parent	abf54564bf62e5476028ef10286748d2e95e7480 [diff]

Merge main -> google (#4048)

* abf54564 Merge google -> main (#4045)
* 1e4349ec Merge branch 'main' into google-to-main
* 1f86f0ef Add new tf.keras.applications models (#3958)
* 41004a91 [spirv] Add a better tiled and vectorized convolution path (#3990)
* c4129446 Bump Tracy to get Ben's dynamic zone colors and to get past (#4042)
* 2a25f5f1 Remove input shape hack from kws tests (#3959)
* 359ffa01 Allow fusing batch_matmul and fill (#4038)
* 35e837ed Merge pull request #4008 from MaheshRavishankar/matmul_fusion
* ffdd9d19 Enable e2e execution of fusion of matmul with producer.
* de116a06 Use tile+fuse on CPU side.
* caf91acb Use fusion of sequence of LinalgOps in IREE SPIR-V codegen path.
* 551f254a Add convert f32 to f16 pass (#4010)
* f1b1e581 Use vector transfer forwarding and dead store elimination in IREE (#4032)
* fac8fd1f Add file to track docker :prod digests (#3955)
* 37f0ae80 Tensorlist refactor to remove shape input and support empty reserve behavior (..
* 7585e532 Add relative_artifacts_dir (#3960)
* da538897 Use subprocess.run and types in manage_images.py (#3947)
* fbc63a4f Upgrade PR action to v3 (#3946)
* 83ea4a3c [llvm] Add a PlanConvLoopOrder pass (#3920)
* 1f1c70a0 Merge pull request #4012
* bab1468c Submodule fix
* bab0308d Merge branch 'main' into google-to-main
* 8df7e5cd Sort add_subdirectory and remove todo given it's fixed now (#4029)
* 1bb131cb Add missing mkdir to fix building and publishing documentation (#4030)
* 46949bad Fix VectorizeMemref out-of-bound access (#4031)
* b32de092 Update Linux docs to include an LLVM AOT example (#3165)
* 40da4547 Update docs to recommend enabling/disabling assertions globally. (#4014)
* fc8da035 Update Tracy to get clean shutdown on Android (#4015)
* 12e20efc support running as root on android. (#4013)
* 1a4f469e Merge branch 'main' into google-to-main
* aff3e9cb Add vectorization support for batch_matmul op (#3997)
* 33d4a6be Changing iree/base/math.h to C and adding some PRNG routines. (#4005)
* 73bdbe7b Fixup EmitC include directories (#3983)
* af7b4c39 Refactor finding python3 to use standard cmake find_package (#3778)
* f92602b7 Always emitting framepointers in generated ELFs (#3987)
* 89ef6328 bump Tracy to get Android fixes (#3988)
* 7f989eb2 Disable MLIR crash reproducer on CI in python tests. (#3943)
* 4db3d08c Adding a demonstration of using the flow dialect post-partitioning. (#3701)
* 589dfa7b Remove no-longer-functional flag (#3961)
* 64543172 Fix MacOS builds after hack-and-slash PR (#3962)
* 8fb887e4 Update links to coverage tables (#3956)
* c93facb4 Adding iree_atomic_slist_t type. (#3917)
* bd082ca6 Merge pull request #3874 from google/benvanik-hack-and-slash
* f4f4ea26 Use UnitTestSpec in tf.keras.layers tests (#3935)
* 7b8e9f75 Reverting flatcc to use our own cmake file for cross-compilation.
* 323e1fde Simplify dylib driver switch.
* 55f3de0d Only register VMLA driver in bindings/java/.
* f429916f Fix warning flag on Windows and HAL CTS driver registration. (#3911)
* 7951e228 Drop IREE_DRIVER_MODULES and iree/base:initializer from ModelBuilder.
* 4e111e2f Disable layering_check in iree/hal/drivers/BUILD.
* 4773736d Add package to iree/base/testing/BUILD.
* 7ca321c1 Skipping dylib driver in simple_embedding_test as a hack.
* 692deb59 Overriding the default MLIR -> LLVM module name.
* 513f40e8 Speculative removing nowindows tags (#3615). If there's something that still d..
* cc47813a Removing the broken forward declarations entirely from some codegen code. http..
* fbcad44d Removing _GNU_SOURCE copt for wait_handle.
* c9b10a01 Fixing bad type in hal/api.h (been there for ages!).
* e0c532ec Changing iree::InitializeEnvironment to iree_flags_parse. Preparation for #3814.
* 8886ac07 Removing iree_api_init from the API.
* 132d747c Removing ALWAYSLINK support from cmake.
* 0ed81f6b Removing iree/base/initializer.h.
* 0135343c Changing to an explicit driver registration mechanism. This is required for ha..
* bf091d3c Removing ALWAYSLINK support from external_cc_library.
* d4bb871d Changing iree-tblgen to not require alwayslink.
* 6bc6f90c Removing IREE_COMMON_INCLUDE_DIRS and uses by LLVM/MLIR.
* 3c082aab Removing IREE_COMMON_INCLUDE_DIRS mhlo pollution.
* 2395cb99 Removing emitc usage of IREE_COMMON_INCLUDE_DIRS for now.
* 036bd966 TODOs on future library layout changes.
* c3a13e62 Rearranging iree/vm/ to reduce a public + cc target.
* 493b0e2b Rearranging iree/base build rules. By moving the dynamic_library_test out of t..
* 8fd38bf9 Replacing uses of some absl utilities now in C++14.
* 67863190 Removing unused absl/types/variant.h include.
* 99bd1af5 Replace absl::exchange with iree::exchange to reduce absl/utility dep.
* c1d0ee10 Removing unused PLATFORM_VULKAN_DEPS. It may be needed internally but it shoul..
* 15437f4b Simplifying iree/hal/dylib/ build config.
* e6984a5a Simplifying iree/hal/ build config.
* 10062814 Simplifying iree/hal/vulkan/ build config.
* 827e51b0 Simplifying iree/hal/llvmjit/ build config.
* 9a72f5d1 Simplifying iree/hal/metal/ build config.
* c7a7d726 Simplifying iree/hal/vmla/ build config.
* 90faf21f Adding IREE_TARGET_GUI_LINKOPTS to remove custom linkopts use.
* e5774c30 Remove unused args from flatbuffers_c_library macro.
* 22d16b4d Adding iree/base/flatcc.h to make flatcc easier to include.
* e44dee56 Switching from -DVK_NO_PROTOTYPES to iree/hal/vulkan/vulkan_headers.h.
* 48ca2fe6 Removing build-config setting of _GNU_SOURCE.
* eeb7dde0 Goodbye flatbuffers (well, the C++ ones anyway).
* 9c676a86 Removing all build config/utils related to flatbuffers.
* 49c61213 byte->ubyte in flatbuffer defs.
* c99000a8 Replacing compiler use of VM bytecode module def flatbuffers->flatcc.
* 1bf1e8d7 Replacing runtime use of metal executable flatbuffers->flatcc. Maybe it works?..
* 48aafb89 Replacing runtime use of spirv executable flatbuffers->flatcc.
* 011e9a2d Replacing runtime use of llvmjit executable flatbuffers->flatcc.
* a021062f Replacing runtime use of dylib executable flatbuffers->flatcc.
* 53a05d73 Replacing runtime use of VMLA executable flatbuffers->flatc.
* 99d30a99 Replacing compiler use of HAL executable flatbuffers->flatc.
* 6ebd1b0c Removing unused tag field in metal/spirv.
* bc685ed7 Adding flatcc json support and making iree-dump-module use it.
* 94b11c35 Adding include for flatcc to flat_c_libraries.
* 1172cf1f Removing unused iree::schemas::reflection_data.
* c86281af Removing unneeded flatbuffers copts.
* 7f3a7e3a Fixing various type warnings. We don't today have these warnings enabled in ou..
* c17659fc Refining MSVC warning set to the minimum required and documenting.
* b7c92bf4 Cleaning up MSVC warnings and syncing with bazel warnings.
* 94356d3b Removing legacy repo_utils.bzl.
* 0f0d9c82 Prevent bazel-to-cmake from running on iree/base/CMakeLists.txt for now.
* 36225a4d Centralizing -ldl/-lpthread linkopts (as they were in bazel already).
* 31c4dbb9 Documenting iree_copts with a nice big warning.
* e4740a57 Pass android libraries as actual linkopts.
* 85cdd868 Fixing cmake style issues - prefer `if(` not `if (` please.
* bf4069e3 Sorting copts/linkopts so we can override things.
* 28040cd8 Simplifying VMA build integration.
* 479ef30f Replacing use of PROJECT_SOURCE_DIR/PROJECT_BINARY_DIR. Those use the previous..

PiperOrigin-RevId: 345252359

tree: ba757c940c6b9010637008d4064e3c64b3370eec

README.md

IREE: Intermediate Representation Execution Environment

IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler that lowers ML models to a unified IR optimized for real-time mobile/edge inference against heterogeneous hardware accelerators. IREE also provides flexible deployment solutions for the compiled ML models.

Project Status

IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!

Communication Channels

GitHub Issues: Preferred for specific technical issues and coordination on upcoming features.
Google IREE Discord Server: The core team and collaborators coordinate daily development here; good for low-latency communication.
Google Groups Email List: Good for general and low-priority discussion.

Related Project Channels

MLIR topic within LLVM Discourse: IREE is enabled by and heavily relies on MLIR. IREE sometimes is referred to in certain MLIR discussions. Useful if you are also interested in MLIR evolution.

Getting Started

For development, IREE supports both Bazel and CMake on Windows and Linux. We are working on enabling macOS support. For deployment, IREE aims to additionally cover Android and iOS.

Please see the Getting Started pages on IREE's documentation hub to configure, compile, and run IREE in your favorite development environment!

Documentation and Talks

IREE hosts all its documentation and project status dashboards on GitHub Pages. We are still building up the website; please feel free to create issues for the documentation you'd like to see!

We also have some public talks that explain IREE's concepts and architecture:

2020-03-18: Interactive HAL IR Walkthrough (Ben Vanik and core team) (recording)
2020-01-31: End-to-end MLIR Workflow in IREE (recording and slides)

Architecture and Goals

IREE adopts a holistic approach towards ML model compilation: the IR produced contains both the scheduling logic, required to communicate data dependencies to low-level parallel pipelined hardware/API like Vulkan, and the execution logic, encoding dense computation on the hardware in the form of hardware/API-specific binaries like SPIR-V.

The architecture of IREE is best illustrated by the following picture:

IREE Architecture

Being compilation-based means IREE does not have a traditional runtime that dispatches “ops” to their fat kernel implementations. What IREE provides is a toolbox for different deployment scenarios. It scales from running generated code on a particular API (such as emitting C code calling external DSP kernels), to a HAL (Hardware Abstraction Layer) that allows the same generated code to target multiple APIs (like Vulkan and Direct3D 12), to a full VM allowing runtime model loading for flexible deployment options and heterogeneous execution.

IREE aims to

Support advanced models on mobile/edge devices. Dynamic shapes, dynamic flow control, dynamic multi-model dispatch, streaming models, tree-based search algorithms, and other are all good examples of exciting ML evolution. We are trying to build IREE from the ground-up to enable these models and run them efficiently on modern hardware, especially on mobile/edge devices.
Demonstrate MLIR‘s ability to develop non-traditional ML compiler backends and runtimes. MLIR enables IREE’s holistic approach of focusing on the math being performed and how that math is scheduled rather than graphs of “ops”.
Embrace standard-based ML via Vulkan. The graphics world is shifting towards favoring modern explicit APIs for performance and predictability and Vulkan is emerging as the “compatibility” layer. We would love to allow hardware vendors to be able to make ML efficient on their hardware without the need for bespoke runtimes and special access. We also would love to let developers and users utilize all the hardware available on as many platforms as possible.

Roadmap and Milestones

IREE is in the early stages of development and not yet ready for broad adoption. Check out the long-term design roadmap to get a sense of where we're headed.

We plan on a quarterly basis using OKRs. Review our latest objectives to get a sense of what we're up to in the near term.

We use GitHub Projects to track progress on IREE components and specific efforts. We use GitHub Milestones to track the work associated with plans for each quarter.

Build Status

CI System	Build System	Platform	Architecture	Component
Kokoro	Bazel	Linux	x86	Core
Kokoro	Bazel	Linux	x86	Bindings
Kokoro	Bazel	Linux	x86-swiftshader	Integrations
Kokoro	Bazel	Linux	x86-turing	Integrations
Kokoro	CMake	Linux	x86-swiftshader	Core + Bindings
Kokoro	CMake	Linux	x86-turing	Core + Bindings
Kokoro	CMake	Android	arm64-v8a	Runtime (build only)
BuildKite	CMake	Android	arm64-v8a	Runtime

License

IREE is licensed under the terms of the Apache license. See LICENSE for more information.