Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 1 | # IREE Releasing |
| 2 | |
| 3 | This file documents the extant release process that IREE uses. This process |
| 4 | and the automation (such as it is) has grown over many years and is due for |
| 5 | a refresh. However, in the interests of documenting what exists, we attempt |
| 6 | to do so here. |
| 7 | |
| 8 | ## Nightly Core Releases |
| 9 | |
| 10 | IREE development is primarily driven via automated nightly release snapshots. |
Scott Todd | 62efaee | 2024-05-31 13:33:55 -0700 | [diff] [blame] | 11 | These are scheduled automatically each day by the |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 12 | `schedule_candidate_release.yml` workflow, which selects a green commit from |
| 13 | main (for non optional CI tasks), created a tag of the format |
Marius Brehler | 6d55a11 | 2024-11-07 21:14:17 +0100 | [diff] [blame] | 14 | `iree-{X.Y.ZrcYYYYMMDD}` and schedules automation to populate the release. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 15 | |
| 16 | The `build_package.yml` workflow then runs jobs to do builds for all |
Scott Todd | 62efaee | 2024-05-31 13:33:55 -0700 | [diff] [blame] | 17 | platforms and packages, finally triggering the |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 18 | `validate_and_publish_release.yml` workflow. |
| 19 | |
| 20 | Release artifacts are uploaded as a GitHub |
| 21 | [pre release](https://github.com/iree-org/iree/releases) and an index of files |
| 22 | is updated by periodic automation at https://iree.dev/pip-release-links.html. |
| 23 | |
Scott Todd | 62efaee | 2024-05-31 13:33:55 -0700 | [diff] [blame] | 24 | Some debugging notes for this process are available here: |
Scott Todd | f190e07 | 2024-09-04 13:12:57 -0700 | [diff] [blame] | 25 | https://iree.dev/developers/debugging/releases/. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 26 | |
| 27 | ### Nightly Release Packages |
| 28 | |
| 29 | A number of packages are produced automatically: |
| 30 | |
| 31 | * `iree-dist-*.tar.xz` (manylinux x86_64 and aarch64): Install image of the |
| 32 | binaries and development assets needed to use or depend on the C/C++ parts |
| 33 | of the project. |
Marius Brehler | b133218 | 2024-11-11 20:00:33 +0100 | [diff] [blame] | 34 | * `iree-base-compiler`: Binary Python wheels |
| 35 | * `iree-base-runtime`: Binary Python wheels |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 36 | * `iree-tools-tf` and `iree-tools-tflite`: Pure Python wheels |
| 37 | |
| 38 | #### Linux Builds |
| 39 | |
Scott Todd | ecc95f7 | 2024-08-26 10:50:44 -0700 | [diff] [blame] | 40 | Binary Linux packages are built using a custom `manylinux` based Docker image |
| 41 | hosted here: |
| 42 | https://github.com/iree-org/base-docker-images/pkgs/container/manylinux_x86_64 |
| 43 | using isolated self-hosted runners (only used for building checked in code) of |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 44 | sufficient size for building large components and GitHub managed runners for |
| 45 | smaller components. The project aims to target all non-EOL Python versions with |
| 46 | Linux builds on x86_64 and aarch64. |
| 47 | |
| 48 | #### Windows Builds |
| 49 | |
Scott Todd | ecc95f7 | 2024-08-26 10:50:44 -0700 | [diff] [blame] | 50 | Windows builds are built using GitHub-hosted runners. Due to the cost, the |
| 51 | project aims to target the most recent version of Python only while building |
| 52 | version N-1 for the first year of the lifecycle of the next version. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 53 | |
Marius Brehler | b133218 | 2024-11-11 20:00:33 +0100 | [diff] [blame] | 54 | Only the Python `iree-base-compiler` and `iree-base-runtime` packages are |
| 55 | built for Windows. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 56 | |
Scott Todd | ecc95f7 | 2024-08-26 10:50:44 -0700 | [diff] [blame] | 57 | The release is published even if the Windows build fails. When this happens, it |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 58 | is fixed forward for the next snapshot. |
| 59 | |
| 60 | #### MacOS Builds |
| 61 | |
Scott Todd | ecc95f7 | 2024-08-26 10:50:44 -0700 | [diff] [blame] | 62 | MacOS builds are performed using GitHub-hosted runners. Due to the cost, the |
| 63 | project aims to target the most recent version of Python only while building |
| 64 | version N-1 for the first year of the lifecycle of the next version. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 65 | |
Marius Brehler | b133218 | 2024-11-11 20:00:33 +0100 | [diff] [blame] | 66 | Only the Python `iree-base-compiler` and `iree-base-runtime` packages are |
| 67 | built for MacOS. |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 68 | |
| 69 | The release is published even if the MacOS build fails. When this happens, it |
| 70 | is fixed forward for the next snapshot. |
| 71 | |
| 72 | ## Retention |
| 73 | |
| 74 | The project will keep pre-release tagged releases on its releases page for a |
| 75 | minimum of 6 months. Releases older than this can be purged. |
| 76 | |
| 77 | ## Distribution to Package Registries |
| 78 | |
| 79 | The following package registry projects are managed as part of the IREE |
| 80 | release process: |
| 81 | |
| 82 | ### PyPI |
| 83 | |
Marius Brehler | b133218 | 2024-11-11 20:00:33 +0100 | [diff] [blame] | 84 | * https://pypi.org/project/iree-base-compiler/ |
| 85 | * https://pypi.org/project/iree-base-runtime/ |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 86 | * https://pypi.org/project/iree-turbine/ |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 87 | * https://pypi.org/project/iree-tools-tf/ |
| 88 | * https://pypi.org/project/iree-tools-tflite/ |
| 89 | |
| 90 | Deprecated projects no longer updated: |
| 91 | |
Marius Brehler | b133218 | 2024-11-11 20:00:33 +0100 | [diff] [blame] | 92 | * https://pypi.org/project/iree-compiler/ (replaced by iree-base-compiler) |
| 93 | * https://pypi.org/project/iree-runtime/ (replaced by iree-base-runtime) |
Stella Laurenzo | afd7cab | 2024-04-21 18:41:28 -0700 | [diff] [blame] | 94 | * https://pypi.org/project/iree-runtime-instrumented/ (functionality is |
| 95 | included in the main iree-runtime package) |
| 96 | * https://pypi.org/project/iree-tools-xla/ (functionality is no longer needed) |
| 97 | |
| 98 | |
| 99 | ## Build Promotion |
| 100 | |
| 101 | There are presently two build promotion processes documented: |
| 102 | |
Scott Todd | db9d92b | 2024-11-07 07:50:04 -0800 | [diff] [blame] | 103 | * Releasing IREE core packages: |
| 104 | https://iree.dev/developers/general/release-management/ |
| 105 | * Releasing iree-turbine packages: |
| 106 | https://github.com/iree-org/iree-turbine/blob/main/docs/releasing.md |