Move packaging out of bindings directory
Creates a new top-level packaging/ directory. We want to ensure that IREE's core components (including bindings) do not depend on tensorflow, and this is also an assumption made by CI setup.
Tested:
Ran all the instructions in the README for both CMake and Bazel on my linux machine.
Closes https://github.com/google/iree/pull/2155
COPYBARA_INTEGRATE_REVIEW=https://github.com/google/iree/pull/2155 from GMNGeoffrey:packaging fb154cc468adca91d876c1100a4feba957ddd75e
PiperOrigin-RevId: 315567516
diff --git a/bindings/python/packaging/README.md b/bindings/python/packaging/README.md
deleted file mode 100644
index 8e9c7cf..0000000
--- a/bindings/python/packaging/README.md
+++ /dev/null
@@ -1,78 +0,0 @@
-# Python packaging scripts.
-
-Note that packages will be placed in `bindings/python/packaging/dist` with the
-canonical instructions. However, the setup scripts can be run from anywhere and
-will create `build` and `dist` directories where run. Wheels can be installed
-with `pip3 install --user dist/*.whl`.
-
-## Building core wheels with CMake
-
-Most of IREE is built/packaged with CMake. For the parts that build with CMake,
-this is preferred.
-
-Canonical instructions follow:
-
-### Linux
-
-```shell
-export LDFLAGS=-fuse-ld=/usr/bin/ld.lld-10
-export PYIREE_CMAKE_BUILD_ROOT=$HOME/build-iree-release
-export IREE_SRC=$HOME/src/iree
-rm -Rf $CMAKE_BUILD_ROOT; mkdir -p $CMAKE_BUILD_ROOT
-cmake -GNinja -B$CMAKE_BUILD_ROOT -H$IREE_SRC \
- -DCMAKE_BUILD_TYPE=Release \
- -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
- -DIREE_BUILD_PYTHON_BINDINGS=ON -DIREE_BUILD_SAMPLES=OFF
-(cd $CMAKE_BUILD_ROOT && ninja)
-(cd $IREE_SRC/bindings/python/packaging && (
-rm -Rf build;
-python3 setup_compiler.py bdist_wheel;
-rm -Rf build;
-python3 setup_rt.py bdist_wheel))
-```
-
-## Building IREE/TensorFlow wheels
-
-If building TensorFlow integration wheels, then this must be done via Bazel. In
-this case, it can be easiest to just package everything from a Bazel build to
-avoid multiple steps.
-
-Canonical instructions follow:
-
-### Env Setup
-
-```shell
-IREE_SRC=$HOME/src/iree
-export PYIREE_BAZEL_BUILD_ROOT="$IREE_SRC/bazel-bin"
-if which cygpath; then
- export PYIREE_BAZEL_BUILD_ROOT="$(cygpath -w "$PYIREE_BAZEL_BUILD_ROOT")"
-fi
-```
-
-### Building:
-
-```shell
-cd $IREE_SRC
-bazel build -c opt \
- //bindings/python/packaging:all_pyiree_packages
-```
-
-# Packaging
-
-```shell
-(cd $IREE_SRC/bindings/python/packaging && (
-rm -Rf build;
-python3 setup_tf.py bdist_wheel))
-```
-
-```shell
-(cd $IREE_SRC/bindings/python/packaging && (
-rm -Rf build;
-python3 setup_compiler.py bdist_wheel))
-```
-
-```shell
-(cd $IREE_SRC/bindings/python/packaging && (
-rm -Rf build;
-python3 setup_rt.py bdist_wheel))
-```
diff --git a/bindings/python/packaging/.gitignore b/packaging/python/.gitignore
similarity index 100%
rename from bindings/python/packaging/.gitignore
rename to packaging/python/.gitignore
diff --git a/bindings/python/packaging/BUILD.bazel b/packaging/python/BUILD.bazel
similarity index 90%
rename from bindings/python/packaging/BUILD.bazel
rename to packaging/python/BUILD.bazel
index 91cd3f8..f95ba18 100644
--- a/bindings/python/packaging/BUILD.bazel
+++ b/packaging/python/BUILD.bazel
@@ -23,9 +23,6 @@
tags = [
# Do not build with ... expansion
"manual",
- # This target does not run on RBE because it depends on a part of
- # TensorFlow that is incompatible with it.
- "nokokoro",
],
deps = [
"//bindings/python:pathsetup", # build_cleaner: keep
diff --git a/packaging/python/README.md b/packaging/python/README.md
new file mode 100644
index 0000000..6fcae98
--- /dev/null
+++ b/packaging/python/README.md
@@ -0,0 +1,82 @@
+# Python packaging scripts.
+
+Note that packages will be placed in `packaging/python/dist` with the canonical
+instructions. However, the setup scripts can be run from anywhere and will
+create `build` and `dist` directories where run. Wheels can be installed with
+`pip3 install --user dist/*.whl`.
+
+## Building core wheels with CMake
+
+Most of IREE is built/packaged with CMake. For the parts that build with CMake,
+this is preferred.
+
+Canonical instructions follow:
+
+### Linux
+
+```shell
+export LDFLAGS=-fuse-ld=/usr/bin/ld.lld
+export PYIREE_CMAKE_BUILD_ROOT="${HOME?}/build-iree-release"
+export IREE_SRC="${HOME?}/src/iree"
+rm -Rf "${PYIREE_CMAKE_BUILD_ROOT?}"; mkdir -p "${PYIREE_CMAKE_BUILD_ROOT?}"
+cmake -GNinja -B"${PYIREE_CMAKE_BUILD_ROOT?}" -H"${IREE_SRC}" \
+ -DCMAKE_BUILD_TYPE=Release \
+ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \
+ -DIREE_BUILD_PYTHON_BINDINGS=ON -DIREE_BUILD_SAMPLES=OFF
+(cd "${PYIREE_CMAKE_BUILD_ROOT?}" && ninja)
+(cd "${IREE_SRC?}/packaging/python" && (
+ rm -Rf build;
+ python3 setup_compiler.py bdist_wheel;
+ rm -Rf build;
+ python3 setup_rt.py bdist_wheel))
+```
+
+## Building IREE/TensorFlow wheels
+
+If building TensorFlow integration wheels, then this must be done via Bazel. In
+this case, it can be easiest to just package everything from a Bazel build to
+avoid multiple steps.
+
+Canonical instructions follow:
+
+### Env Setup
+
+```shell
+IREE_SRC=$HOME/src/iree
+export PYIREE_BAZEL_BUILD_ROOT="$IREE_SRC/bazel-bin"
+if which cygpath; then
+ export PYIREE_BAZEL_BUILD_ROOT="$(cygpath -w "$PYIREE_BAZEL_BUILD_ROOT")"
+fi
+```
+
+### Building:
+
+Optionally add: `--define=PYIREE_TF_DISABLE_KERNELS=1` to build a 'thin' (less
+functional) version without TensorFlow kernels. This should not be done for
+released binaries but can help while developing.
+
+```shell
+cd $IREE_SRC
+bazel build -c opt \
+ //packaging/python:all_pyiree_packages
+```
+
+# Packaging
+
+```shell
+(cd $IREE_SRC/packaging/python && (
+ rm -Rf build;
+ python3 setup_tf.py bdist_wheel))
+```
+
+```shell
+(cd $IREE_SRC/packaging/python && (
+ rm -Rf build;
+ python3 setup_compiler.py bdist_wheel))
+```
+
+```shell
+(cd $IREE_SRC/packaging/python && (
+ rm -Rf build;
+ python3 setup_rt.py bdist_wheel))
+```
diff --git a/bindings/python/packaging/__init__.py b/packaging/python/__init__.py
similarity index 100%
rename from bindings/python/packaging/__init__.py
rename to packaging/python/__init__.py
diff --git a/bindings/python/packaging/common_setup.py b/packaging/python/common_setup.py
similarity index 95%
rename from bindings/python/packaging/common_setup.py
rename to packaging/python/common_setup.py
index 8fc0f4a..a34f493 100644
--- a/bindings/python/packaging/common_setup.py
+++ b/packaging/python/common_setup.py
@@ -32,7 +32,7 @@
if cmake_build_root and bazel_build_root:
print("ERROR: Both PYIREE_CMAKE_BUILD_ROOT and PYIREE_BAZEL_BUILD_ROOT"
- "cannot be set at the same time")
+ " cannot be set at the same time")
sys.exit(1)
if cmake_build_root:
@@ -46,13 +46,13 @@
# Find the path to the runfiles of the built target:
# //bindings/python/packaging:all_pyiree_packages
runfiles_dir = os.path.join(
- bazel_build_root, "bindings", "python", "packaging",
+ bazel_build_root, "packaging", "python",
"all_pyiree_packages%s.runfiles" % (get_exe_suffix(),))
if not os.path.isdir(runfiles_dir):
print("ERROR: Could not find build target 'all_pyiree_packages':",
runfiles_dir)
print("Make sure to build target",
- "//bindings/python/packaging:all_pyiree_packages")
+ "//packaging/python:all_pyiree_packages")
sys.exit(1)
# And finally seek into the corresponding path in the runfiles dir.
# Aren't bazel paths fun???
diff --git a/bindings/python/packaging/dummy_exclude_from_package.py b/packaging/python/dummy_exclude_from_package.py
similarity index 100%
rename from bindings/python/packaging/dummy_exclude_from_package.py
rename to packaging/python/dummy_exclude_from_package.py
diff --git a/bindings/python/packaging/setup_compiler.py b/packaging/python/setup_compiler.py
similarity index 100%
rename from bindings/python/packaging/setup_compiler.py
rename to packaging/python/setup_compiler.py
diff --git a/bindings/python/packaging/setup_rt.py b/packaging/python/setup_rt.py
similarity index 89%
rename from bindings/python/packaging/setup_rt.py
rename to packaging/python/setup_rt.py
index 1b0ab0c..ef39248 100644
--- a/bindings/python/packaging/setup_rt.py
+++ b/packaging/python/setup_rt.py
@@ -16,11 +16,6 @@
# Build platform specific wheel files for the pyiree.rt package.
# Built artifacts are per-platform and build out of the build tree.
-# Usage:
-# ------
-# Windows with CMake:
-# export CMAKE_BUILD_ROOT='D:\src\build-iree' # Must be native path
-# python ./setup_rt.py bdist_wheel
import os
import setuptools
diff --git a/bindings/python/packaging/setup_tf.py b/packaging/python/setup_tf.py
similarity index 72%
rename from bindings/python/packaging/setup_tf.py
rename to packaging/python/setup_tf.py
index 9b1432d..6f97d70 100644
--- a/bindings/python/packaging/setup_tf.py
+++ b/packaging/python/setup_tf.py
@@ -16,20 +16,6 @@
# Build platform specific wheel files for the pyiree.tf packages.
# Built artifacts are per-platform and build out of the build tree.
-# Usage:
-# ------
-# bazel build -c opt //integrations/tensorflow/bindings/python/packaging:all_tf_packages
-# python3 ./setup_tf.py bdist_wheel
-#
-# Tips:
-# Optionally add: --define=PYIREE_TF_DISABLE_KERNELS=1
-# to build a 'thin' (less functional) version without TensorFlow kernels.
-# This should not be done for released binaries but can help while developing.
-#
-# Note that this script violates our general policy of keeping TensorFlow
-# things in the integrations/tensorflow directory because it is more convenient
-# to have all packaging scripts in one place, and this will not grow further
-# dependencies.
import os
import platform
