[Infra] Initial Migration off GCP Runners (#18381)
This commit is part of this larger issue that is tracking our migration
off the GCP runners, storage buckets, etc:
https://github.com/iree-org/iree/issues/18238.
In this initial port, we move over one high traffic job
(`linux_x86_64_release_packages`) and a few nightlies
(`linux_x64_clang_tsan`, `linux_x64_clang_debug`) to monitor and make
sure the cluster is working as intended.
Time Comparisons:
Job | Cache? | Runner cluster | Time | Logs
-- | -- | -- | -- | --
linux_x86_64_release_packages | GitHub Cache | AKS Cluster | 9 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10797464301/job/29948809708)
linux_x64_clang_tsan | GCP Cache | AKS cluster | 10 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10797464292/job/29948816896)
linux_x64_clang_debug | GCP Cache | AKS cluster | 11 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10797464308/job/29948805561)
linux_x64_clang_tsan | No Cache | AKS cluster | 17 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10798471545/job/29952051686)
linux_x64_clang_debug | No Cache | AKS cluster | 13 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10798475582/job/29952064138)
| | |
linux_x86_64_release_packages | GitHub Cache | GCP Runners | 11 minutes
|
[logs](https://github.com/iree-org/iree/actions/runs/10796348911/job/29945148145)
linux_x64_clang_tsan | GCP Cache | GCP Runners | 14 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10789692182/job/29923234380)
linux_x64_clang_debug | GCP Cache | GCP Runners | 15 minutes |
[logs](https://github.com/iree-org/iree/actions/runs/10680250213/job/29601266656)
The GCP cache timings for the AKS cluster are not a great representation
of what we will be seeing going forward because the AKS cluster does not
have the setup/authentication to write to the GCP cache. We have changes
coming in
https://github.com/iree-org/iree/tree/shared/runner-cluster-migration
that will spin up an Azure cache using sccache to help with the No Cache
timings. Right now the cluster is using 96 core machines, which we can
probably tone down when the caching work lands.
---------
Signed-off-by: saienduri <saimanas.enduri@amd.com>
diff --git a/.github/workflows/ci_linux_x64_clang_debug.yml b/.github/workflows/ci_linux_x64_clang_debug.yml
index 64b04ad..ddc25aa 100644
--- a/.github/workflows/ci_linux_x64_clang_debug.yml
+++ b/.github/workflows/ci_linux_x64_clang_debug.yml
@@ -33,12 +33,7 @@
# so run on self-hosted CPU build runners instead.
linux_x64_clang_debug:
needs: setup
- runs-on:
- - self-hosted # must come first
- - runner-group=${{ needs.setup.outputs.runner-group }}
- - environment=${{ needs.setup.outputs.runner-env }}
- - cpu
- - os-family=Linux
+ runs-on: azure-linux-scale
container: ghcr.io/iree-org/cpubuilder_ubuntu_jammy_x86_64@sha256:54d9d17a79caa083aeff1243b27e767df2629b533c26e0b65d41beb160f197e4
defaults:
run:
diff --git a/.github/workflows/ci_linux_x64_clang_tsan.yml b/.github/workflows/ci_linux_x64_clang_tsan.yml
index bd3c6ab..63be367 100644
--- a/.github/workflows/ci_linux_x64_clang_tsan.yml
+++ b/.github/workflows/ci_linux_x64_clang_tsan.yml
@@ -31,12 +31,7 @@
linux_x64_clang_tsan:
needs: setup
- runs-on:
- - self-hosted # must come first
- - runner-group=${{ needs.setup.outputs.runner-group }}
- - environment=${{ needs.setup.outputs.runner-env }}
- - cpu
- - os-family=Linux
+ runs-on: azure-linux-scale
steps:
- name: "Checking out repository"
uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
diff --git a/.github/workflows/pkgci_build_packages.yml b/.github/workflows/pkgci_build_packages.yml
index 366716c..16ed708 100644
--- a/.github/workflows/pkgci_build_packages.yml
+++ b/.github/workflows/pkgci_build_packages.yml
@@ -15,12 +15,7 @@
jobs:
linux_x86_64_release_packages:
name: Linux Release (x86_64)
- runs-on:
- - self-hosted # must come first
- - runner-group=${{ github.event_name == 'pull_request' && 'presubmit' || 'postsubmit' }}
- - environment=prod
- - cpu
- - os-family=Linux
+ runs-on: azure-linux-scale
strategy:
fail-fast: false
env:
