Test the bring-your-own-LLVM path. (#14035)

* Adds a`byo_llvm` CI job. Do we want to disable running it on PRs? It's
enabled for now so we get live coverage while iterating on this PR but
we can always flip that.
* Note: the current CI job was the first to have ccache enabled so it
didn't benefit from ccache hits; subsequent ones should be cheaper (this
initial one took 15 minutes).
* Changes to the (user-facing) `byo_llvm.sh` script:
* Canonicalize paths to absolute paths. These end up in CMake variables
like `CMAKE_MODULE_PATH` where relative paths are a footgun.
* Defaults to buiding the `llvm-cpu` target backend. Users can still
disable it, but at the moment about 20 tests fail without it enabled
(filed #14034). Not a problem for users unless they run IREE tests.
Until that's fixed, enabling that here allows to keep the CI job aligned
with the default, and increases coverage (otherwise, users who do enable
`llvm-cpu` would be outside of CI coverage).
* Changes both the name and the default values of the environment
variables controlling build/install dirs. **Caveat existing users, this
could break incremental rebuilds -- as this is moving to a new
directory, it will lose existing values of CMake variables, etc.**
* `LLVM_BUILD_DIR` changed to `IREE_BYOLLVM_BUILD_DIR` because that's
only the parent directory under which we create `llvm/`, `mlir/`,
`iree/` subdirs. The former name made it sound like it was the `llvm/`
build dir itself.
* Default value `iree-llvm-build` likewise changed to
`iree-byollvm-build`.
* Changes to general CMake code:
* Continuing along the lines of earlier `byo_llvm` fixes (#13968) which
introduced `IREE_CLANG_BINARY` to refer to the `clang` binary regardless
of whether it's a CMake target (which is only the case in the bundled
build, not in `byo_llvm`), running tests revealed that we needed the
same for `lld`, so here comes `IREE_LLD_BINARY`, so we can execute `lld`
even if we don't have a `IREE_LLD_TARGET`.
* The one part of the code that decidedly doesn't currently work with
`byo_llvm` is the linking part of the Compiler API, because it relies on
linking against LLD libraries, and for that it currently requires
IREE_LLD_TARGET. I don't know how easy or feasible that would be to
overcome (though that might be easy if those LLD libraries are exported
by the LLVM install).
* Changes to tests:
* `assign_target_devices.mlir` was assuming the `vulkan-spirv` backend
to be enabled.
* A number of tests, mainly tests that depend on the Compiler API, are
skipped when `IREE_LLD_TARGET` is not defined.

Fixes #13989
14 files changed
tree: 4806d771c05151b354b043ed32517841e81bd6b4
  1. .devcontainer/
  2. .github/
  3. build_tools/
  4. compiler/
  5. docs/
  6. experimental/
  7. integrations/
  8. lib/
  9. llvm-external-projects/
  10. runtime/
  11. samples/
  12. tests/
  13. third_party/
  14. tools/
  15. .bazel_to_cmake.cfg.py
  16. .bazelignore
  17. .bazelrc
  18. .bazelversion
  19. .clang-format
  20. .dockerignore
  21. .gitignore
  22. .gitmodules
  23. .pylintrc
  24. .style.yapf
  25. .yamllint.yml
  26. AUTHORS
  27. BUILD.bazel
  28. CITATION.cff
  29. CMakeLists.txt
  30. configure_bazel.py
  31. CONTRIBUTING.md
  32. LICENSE
  33. README.md
  34. WORKSPACE
README.md

IREE: Intermediate Representation Execution Environment

IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.

See our website for project details, user guides, and instructions on building from source.

CI Status

Project Status

IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!

Communication Channels

Related Project Channels

  • MLIR topic within LLVM Discourse: IREE is enabled by and heavily relies on MLIR. IREE sometimes is referred to in certain MLIR discussions. Useful if you are also interested in MLIR evolution.

Architecture Overview

IREE Architecture IREE Architecture

See our website for more information.

Presentations and Talks

  • 2021-06-09: IREE Runtime Design Tech Talk (recording and slides)
  • 2020-08-20: IREE CodeGen: MLIR Open Design Meeting Presentation (recording and slides)
  • 2020-03-18: Interactive HAL IR Walkthrough (recording)
  • 2020-01-31: End-to-end MLIR Workflow in IREE: MLIR Open Design Meeting Presentation (recording and slides)

License

IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.