Merge "Python bindings and importers" page into "getting started". (#13462)
The `iree-import-tf` and `iree-import-tflite` tools are pure Python now,
requiring no Bazel build in IREE. Thanks to this, the instructions for
how to build the Python bindings from source (including framework
importers) can be simplified enough that they can be merged into the
main "getting started" page.
Fixes https://github.com/openxla/iree/issues/13037.
### Website links
| | Getting Started | Python bindings and importers |
|--------|--------|--------|
| Old | [live
site](https://openxla.github.io/iree/building-from-source/getting-started/)
| [live
site](https://openxla.github.io/iree/building-from-source/python-bindings-and-importers/)
|
| New (this PR) |
[preview](https://scotttodd.github.io/iree/building-from-source/getting-started/)
| N/A |
skip-ci: documentation/comment only changes
diff --git a/build_tools/scripts/get_e2e_artifacts.py b/build_tools/scripts/get_e2e_artifacts.py
index 72f5f20..634ee31 100755
--- a/build_tools/scripts/get_e2e_artifacts.py
+++ b/build_tools/scripts/get_e2e_artifacts.py
@@ -150,8 +150,7 @@
print(
"The bazel integrations build and tests are deprecated. This script "
"may be reworked in the future. For the time being refer to "
- "https://openxla.github.io/iree/building-from-source/python-bindings-and-importers/ "
- "and https://github.com/openxla/iree/blob/main/docs/developers/developing_iree/e2e_benchmarking.md "
+ "https://github.com/openxla/iree/blob/main/docs/developers/developing_iree/e2e_benchmarking.md "
"for information on how to run TensorFlow benchmarks.")
exit(1)
diff --git a/docs/developers/developing_iree/e2e_benchmarking.md b/docs/developers/developing_iree/e2e_benchmarking.md
index cb18a5e..55b5098 100644
--- a/docs/developers/developing_iree/e2e_benchmarking.md
+++ b/docs/developers/developing_iree/e2e_benchmarking.md
@@ -14,7 +14,7 @@
to, and to run them using valid inputs for each model.
This guide assumes that you can run the tensorflow integration tests. See
-[this doc](https://openxla.github.io/iree/building-from-source/python-bindings-and-importers/)
+[this doc](https://openxla.github.io/iree/building-from-source/getting-started/#python-bindings)
for more information. That doc also covers writing new tests, which you'll need
to do if you'd like to benchmark a new TensorFlow model.
diff --git a/docs/website/docs/bindings/python.md b/docs/website/docs/bindings/python.md
index c458fc4..0befe68 100644
--- a/docs/website/docs/bindings/python.md
+++ b/docs/website/docs/bindings/python.md
@@ -22,9 +22,9 @@
Collectively, these packages allow for importing from frontends, compiling
towards various targets, and executing compiled code on IREE's backends.
-!!! Caution
- The TensorFlow, TensorFlow Lite, and XLA packages are currently only
- available on Linux and macOS. They are not available on Windows yet (see
+!!! Caution "Caution - Operating system support"
+ Packages are currently only available on Linux and macOS. They are not
+ available on Windows yet (see
[this issue](https://github.com/openxla/iree/issues/6417)).
## Prerequisites
@@ -33,7 +33,7 @@
[Python 3](https://www.python.org/downloads/) and
[pip](https://pip.pypa.io/en/stable/installing/), as needed.
-???+ tip
+???+ Tip "Tip - Virtual environments"
We recommend using virtual environments to manage python packages, such as
through `venv`
([about](https://docs.python.org/3/library/venv.html),
@@ -62,15 +62,6 @@
When done, run `deactivate`.
-<!-- TODO(??): use setup.py install_requires for any dependencies IREE needs -->
-
-Next, install packages:
-
-``` shell
-python -m pip install --upgrade pip
-python -m pip install numpy absl-py
-```
-
## Installing IREE packages
### Prebuilt packages
@@ -101,11 +92,11 @@
iree-tools-xla
```
-!!! Tip
+!!! Tip "Tip - Nightly releases"
- Nightly packages are also published on
+ Unstable packages are also published nightly on
[GitHub releases](https://github.com/openxla/iree/releases). To use these,
- run `pip install` with this extra option:
+ run `pip install` with this option:
```
--find-links https://openxla.github.io/iree/pip-release-links.html
@@ -113,7 +104,7 @@
### Building from source
-See [Building Python bindings](../../building-from-source/python-bindings-and-importers/#building-python-bindings)
+See [Building Python bindings](../../building-from-source/getting-started/#python-bindings)
page for instructions for building from source.
## Using the Python bindings
diff --git a/docs/website/docs/building-from-source/getting-started.md b/docs/website/docs/building-from-source/getting-started.md
index 46b08d8..668d8ad 100644
--- a/docs/website/docs/building-from-source/getting-started.md
+++ b/docs/website/docs/building-from-source/getting-started.md
@@ -1,6 +1,6 @@
# Getting started
-## Prerequisites
+## :octicons-download-16: Prerequisites
IREE can be built from source using [CMake](https://cmake.org/). We also
recommend the [Ninja](https://ninja-build.org/) CMake generator and the
@@ -56,7 +56,7 @@
<!-- TODO(#12921): add notes about Docker and/or dev containers -->
-## Quickstart: clone and build
+## :octicons-rocket-16: Quickstart: clone and build
Use [Git](https://git-scm.com/) to clone the IREE repository and initialize its
submodules:
@@ -85,7 +85,7 @@
Use case permitting, disabling the compiler build with
`-DIREE_BUILD_COMPILER=OFF` will drastically simplify the build.
-## Configuration settings
+## :octicons-sliders-16: Configuration settings
The configure step should be customized for your build environment. These
settings can improve compile and link times substantially.
@@ -148,7 +148,7 @@
See also our [developer documentation for ccache](https://github.com/openxla/iree/blob/main/docs/developers/developing_iree/ccache.md).
-### Optional components
+### :octicons-gear-16: Optional components
By default, the CMake build includes:
@@ -166,7 +166,7 @@
These can be changed via the `IREE_` CMake options listed in the root
[`CMakeLists.txt`](https://github.com/openxla/iree/blob/main/CMakeLists.txt).
-### Extensions and project integrations
+### Extensions and integrations
When using IREE within other projects, you can register compiler plugins and
runtime HAL drivers. You can also bring your own copy of LLVM and some other
@@ -174,7 +174,7 @@
[`CMakeLists.txt`](https://github.com/openxla/iree/blob/main/CMakeLists.txt)
for details.
-## Tests and samples
+## :octicons-code-16: Tests and samples
### Running tests
@@ -245,3 +245,145 @@
../iree-build/tools/iree-compile --help
../iree-build/tools/iree-run-module --help
```
+
+## :simple-python: Python bindings
+
+Python packages can either be built from source or installed from our releases.
+See the [Python bindings](../bindings/python.md) page for details about the
+bindings themselves.
+
+### Dependencies
+
+You will need a recent Python installation >=3.8 (we aim to support
+[non-eol Python versions](https://endoflife.date/python)).
+
+???+ Tip "Tip - Managing Python versions"
+ Make sure your 'python' is what you expect:
+
+ === "Linux"
+
+ Note that on multi-python systems, this may have a version suffix, and on
+ many Linuxes where python2 and python3 can co-exist, you may also want to
+ use `python3`.
+
+ ``` shell
+ which python
+ python --version
+ ```
+
+ === "macOS"
+
+ Note that on multi-python systems, this may have a version suffix, and on
+ macOS where python2 and python3 can co-exist, you may also want to use `python3`.
+
+ ``` shell
+ which python
+ python --version
+ ```
+
+ === "Windows"
+ The
+ [Python launcher for Windows](https://docs.python.org/3/using/windows.html#python-launcher-for-windows) (`py`) can help manage versions.
+
+ ``` powershell
+ which python
+ python --version
+ py --list-paths
+ ```
+
+???+ Tip "Tip - Virtual environments"
+ We recommend using virtual environments to manage python packages, such as
+ through `venv`
+ ([about](https://docs.python.org/3/library/venv.html),
+ [tutorial](https://docs.python.org/3/tutorial/venv.html)):
+
+ === "Linux"
+
+ ``` shell
+ python -m venv .venv
+ source .venv/bin/activate
+ ```
+
+ === "macOS"
+
+ ``` shell
+ python -m venv .venv
+ source .venv/bin/activate
+ ```
+
+ === "Windows"
+
+ ``` powershell
+ python -m venv .venv
+ .venv\Scripts\activate.bat
+ ```
+
+ When done, run `deactivate`.
+
+``` shell
+# Upgrade PIP before installing other requirements
+python -m pip install --upgrade pip
+
+# Install IREE build requirements
+python -m pip install -r runtime/bindings/python/iree/runtime/build_requirements.txt
+```
+
+### Building with CMake
+
+To build the Python bindings, configure CMake with the
+`IREE_BUILD_PYTHON_BINDINGS` option. We also recommend explicitly setting which
+Python executable to use with `Python3_EXECUTABLE`:
+
+``` shell
+# Configure (including other options as discussed above)
+cmake -G Ninja -B ../iree-build/ \
+ -DIREE_BUILD_PYTHON_BINDINGS=ON \
+ -DPython3_EXECUTABLE="$(which python)" \
+ .
+
+# Build
+cmake --build ../iree-build/
+```
+
+### Using the Python bindings
+
+Extend your `PYTHONPATH` with IREE's `bindings/python` paths and try importing:
+
+=== "Linux"
+
+ ``` shell
+ source ../iree-build/.env && export PYTHONPATH
+ python -c "import iree.compiler"
+ python -c "import iree.runtime"
+ ```
+
+=== "macOS"
+
+ ``` shell
+ source ../iree-build/.env && export PYTHONPATH
+ python -c "import iree.compiler"
+ python -c "import iree.runtime"
+ ```
+
+=== "Windows"
+
+ ``` powershell
+ ../iree-build/.env.bat
+ python -c "import iree.compiler"
+ python -c "import iree.runtime"
+ ```
+
+Using IREE's ML framework importers requires a few extra steps:
+
+``` shell
+# Install test requirements
+python -m pip install -r integrations/tensorflow/test/requirements.txt
+
+# Install pure Python packages (no build required)
+python -m pip install integrations/tensorflow/python_projects/iree_tf
+python -m pip install integrations/tensorflow/python_projects/iree_tflite
+
+# Then test the tools:
+iree-import-tf --help
+iree-import-tflite --help
+```
diff --git a/docs/website/docs/building-from-source/index.md b/docs/website/docs/building-from-source/index.md
index 2528f57..3d0a727 100644
--- a/docs/website/docs/building-from-source/index.md
+++ b/docs/website/docs/building-from-source/index.md
@@ -9,7 +9,6 @@
## Reference pages
* [Getting started](./getting-started.md)
-* [Bindings and importers](./python-bindings-and-importers.md)
* [Android cross-compilation](./android.md)
* [iOS cross-compilation](./ios.md)
* [RISC-V cross-compilation](./riscv.md)
diff --git a/docs/website/docs/building-from-source/python-bindings-and-importers.md b/docs/website/docs/building-from-source/python-bindings-and-importers.md
deleted file mode 100644
index 06e51a7..0000000
--- a/docs/website/docs/building-from-source/python-bindings-and-importers.md
+++ /dev/null
@@ -1,211 +0,0 @@
-# Python bindings and importers
-
-!!! Attention
- These components are more complicated to build from source than the rest of
- the project. If your usage does not require making source changes, prefer to
- [install from the official binary distributions](../bindings/python.md#installing-iree-packages)
- instead.
-
-This page covers how to build IREE's Python-based bindings and import tools from
-source. These components are built using CMake as well as other dependencies and
-each section extends the basic build steps in the
-[getting started](./getting-started.md) page.
-
-## Building Python bindings
-
-This section describes how to build and interactively use built-from-source
-Python bindings for the following packages:
-
-| Python Import | Description |
-|------------------------|------------------------------------------------|
-| `import iree.compiler` | IREE's generic compiler tools and helpers |
-| `import iree.runtime` | IREE's runtime, including CPU and GPU backends |
-
-Also see [instructions for installing pre-built binaries](../bindings/python.md).
-
-**Pre-requisites:**
-
-* A relatively recent Python3 installation >=3.8 (we aim to support
- [non-eol Python versions](https://endoflife.date/python)).
-
-**CMake Variables:**
-
-* **`IREE_BUILD_PYTHON_BINDINGS`** : `BOOL`
-
- Enables building of Python bindings under `runtime/bindings/python` in the
- repository. Defaults to `OFF`.
-
-* **`Python3_EXECUTABLE`** : `PATH`
-
- Full path to the Python3 executable to build against. If not specified, CMake
- will auto-detect this, which often yields incorrect results on systems
- with multiple Python versions. Explicitly setting this is recommended.
- Note that mixed case of the option.
-
-### Environment setup
-
-We recommend using virtual environments to manage python packages, such
-as through `venv`, which may need to be installed via your system
-package manager ([about](https://docs.python.org/3/library/venv.html),
-[tutorial](https://docs.python.org/3/tutorial/venv.html)):
-
-=== "Linux"
-
- ``` shell
- # Make sure your 'python' is what you expect. Note that on multi-python
- # systems, this may have a version suffix, and on many Linuxes where
- # python2 and python3 can co-exist, you may also want to use `python3`.
- which python
- python --version
-
- # Create a persistent virtual environment (first time only).
- python -m venv iree.venv
-
- # Activate the virtual environment (per shell).
- # Now the `python` command will resolve to your virtual environment
- # (even on systems where you typically use `python3`).
- source iree.venv/bin/activate
-
- # Upgrade PIP. On Linux, many packages cannot be installed for older
- # PIP versions. See: https://github.com/pypa/manylinux
- python -m pip install --upgrade pip
-
- # Install IREE build pre-requisites.
- python -m pip install -r ./runtime/bindings/python/iree/runtime/build_requirements.txt
- ```
-
-=== "macOS"
-
- ``` shell
- # Make sure your 'python' is what you expect. Note that on multi-python
- # systems, this may have a version suffix, and on macOS where python2
- # and python3 can co-exist, you may also want to use `python3`.
- which python
- python --version
-
- # Create a persistent virtual environment (first time only).
- python -m venv iree.venv
-
- # Activate the virtual environment (per shell).
- # Now the `python` command will resolve to your virtual environment
- # (even on systems where you typically use `python3`).
- source iree.venv/bin/activate
-
- # Upgrade PIP.
- python -m pip install --upgrade pip
-
- # Install IREE build pre-requisites.
- python -m pip install -r ./runtime/bindings/python/iree/runtime/build_requirements.txt
- ```
-
-=== "Windows"
-
- ``` powershell
- # Make sure your 'python' is what you expect.
- # Also consider using the Python launcher 'py' instead of 'python':
- # https://docs.python.org/3/using/windows.html#python-launcher-for-windows
- which python
- python --version
- py --list-paths
-
- # Create a persistent virtual environment (first time only).
- python -m venv .venv
-
- # Activate the virtual environment (per shell).
- # Now the `python` command will resolve to your virtual environment
- # (even on systems where you typically use `python3`).
- .venv\Scripts\activate.bat
-
- # Upgrade PIP.
- python -m pip install --upgrade pip
-
- # Install IREE build pre-requisites.
- python -m pip install -r runtime\bindings\python\iree\runtime\build_requirements.txt
- ```
-
-When you are done with the venv, you can close it by closing your shell
-or running `deactivate`.
-
-### Building with CMake
-
-From the `iree-build` directory:
-
-=== "Linux"
-
- ``` shell
- cmake \
- -GNinja \
- -DCMAKE_BUILD_TYPE=RelWithDebInfo \
- -DIREE_BUILD_PYTHON_BINDINGS=ON \
- -DPython3_EXECUTABLE="$(which python)" \
- .
- cmake --build .
-
- # Add the bindings/python paths to PYTHONPATH and use the API.
- source .env && export PYTHONPATH
- python -c "import iree.compiler"
- python -c "import iree.runtime"
- ```
-
-=== "macOS"
-
- ``` shell
- cmake \
- -GNinja \
- -DCMAKE_BUILD_TYPE=RelWithDebInfo \
- -DIREE_BUILD_PYTHON_BINDINGS=ON \
- -DPython3_EXECUTABLE="$(which python)" \
- .
- cmake --build .
-
- # Add the bindings/python paths to PYTHONPATH and use the API.
- source .env && export PYTHONPATH
- python -c "import iree.compiler"
- python -c "import iree.runtime"
- ```
-
-=== "Windows"
-
- ``` powershell
- cmake -GNinja -DCMAKE_BUILD_TYPE=RelWithDebInfo -DIREE_BUILD_PYTHON_BINDINGS=ON .
- cmake --build .
-
- # Add the bindings/python paths to PYTHONPATH and use the API.
- .env.bat
- python -c "import iree.compiler"
- python -c "import iree.runtime"
- ```
-
-Tests can now be run individually via python or via ctest.
-
-## Building TensorFlow frontend bindings
-
-This section describes how to build compiler tools and Python bindings for
-importing models from various TensorFlow-ecosystem frontends, including
-TensorFlow, XLA (used for JAX), and TFLite. It extends the instructions in
-[Building Python Bindings](#building-python-bindings) above with additional
-steps that are TensorFlow specific. There are various ways to achieve these
-ends, but this section describes the canonical method that the core
-developers recommend. Upon completing these steps, you will have access to
-additional Python packages:
-
-| Python Import | Description |
-|-------------------------------------|-----------------------------------------------------------------------------|
-| `import iree.compiler.tools.tf` | Tools for importing from [TensorFlow](https://www.tensorflow.org/) |
-| `import iree.compiler.tools.tflite` | Tools for importing from [TensorFlow Lite](https://www.tensorflow.org/lite) |
-| `import iree.compiler.tools.xla` | Tools for importing from [XLA](https://www.tensorflow.org/xla) |
-
-These tools packages are needed in order for the frontend specific, high-level
-APIs under `import iree.compiler.tf`, `import iree.compiler.tflite`,
-`import iree.compiler.xla`, and `import iree.jax` to be fully functional.
-
-!!! Caution
-
- This section is under construction. Refer to the
- [source documentation](https://github.com/openxla/iree/tree/main/integrations/tensorflow#readme)
- for the latest building from source instructions.
-
-???+ Note
- Due to the difficulties using Bazel and compiling TensorFlow, only
- compilation on Linux with clang is supported. Other OS's and compilers are
- "best effort" with patches to improve support welcome.
diff --git a/docs/website/docs/getting-started/pytorch.md b/docs/website/docs/getting-started/pytorch.md
index f5f1ea7..ff9aa96 100644
--- a/docs/website/docs/getting-started/pytorch.md
+++ b/docs/website/docs/getting-started/pytorch.md
@@ -7,7 +7,7 @@
## Prerequisites
Install IREE pip packages, either from pip or by
-[building from source](../building-from-source/python-bindings-and-importers.md):
+[building from source](../building-from-source/getting-started/#python-bindings):
```shell
pip install \
diff --git a/docs/website/docs/getting-started/tensorflow.md b/docs/website/docs/getting-started/tensorflow.md
index beea783..1c6476f 100644
--- a/docs/website/docs/getting-started/tensorflow.md
+++ b/docs/website/docs/getting-started/tensorflow.md
@@ -17,7 +17,7 @@
```
Install IREE pip packages, either from pip or by
-[building from source](../building-from-source/python-bindings-and-importers.md):
+[building from source](../building-from-source/getting-started/#python-bindings):
```shell
python -m pip install \
diff --git a/docs/website/mkdocs.yml b/docs/website/mkdocs.yml
index c73ac56..e29c97b 100644
--- a/docs/website/mkdocs.yml
+++ b/docs/website/mkdocs.yml
@@ -117,7 +117,6 @@
- "Building from source":
- "building-from-source/index.md"
- "building-from-source/getting-started.md"
- - "building-from-source/python-bindings-and-importers.md"
- "building-from-source/android.md"
- "building-from-source/ios.md"
- "building-from-source/riscv.md"
diff --git a/tests/e2e/models/mnist_train_test/README.md b/tests/e2e/models/mnist_train_test/README.md
index 25bf579..d6f393d 100644
--- a/tests/e2e/models/mnist_train_test/README.md
+++ b/tests/e2e/models/mnist_train_test/README.md
@@ -8,7 +8,7 @@
python -m venv generate_mnist.venv
source generate_mnist.venv/bin/activate
# Add IREE Python to your PYTHONPATH, following
-# https://openxla.github.io/iree/building-from-source/python-bindings-and-importers
+# https://openxla.github.io/iree/building-from-source/getting-started/#python-bindings
pip install -r generate_test_data_requirements.txt
python ./generate_test_data.py
```
