icon: octicons/check-circle-16

GitHub Actions

:octicons-book-16: Overview

We use GitHub Actions for continuous automation (CI) and continuous delivery (CD) workflows:

  • Code formatting and linting.
  • Building from source and running tests.
  • Building packages for testing and releases.
  • Testing packages across a variety of platforms.
  • Publishing the https://iree.dev website.
  • Updating dependencies.

Workflows are defined directly in the repository at .github/workflows/. We use a mix of GitHub-hosted runners and self-hosted runners to get automated build and test coverage across a variety of platforms and hardware accelerators.

Terminology primer

(Read more on https://docs.github.com/en/actions/learn-github-actions/understanding-github-actions)

  • Workflows are configurable automated processes that run one or more jobs.
  • Jobs are a set of steps in a workflow that are executed on the same runner.
  • Steps are lists of commands or meta actions to run in a shell environment.
  • Runners are servers (physical or virtual machines) that run workflows when triggered.
  • Events are specific activities in a repository that trigger a workflow run.
graph
  accTitle: Example workflow run diagram
  accDescr {
    An event runs two jobs - job 1 on runner 1 and job 2 on runner.
    Job 1 runs four steps, each either an action or script.
    Job 2 runs three other steps.
  }

  event("Event")

  event --> runner_1
  event --> runner_2

  subgraph runner_1["Runner 1"]
    job_1("Job 1
    • Step 1: Run action
    • Step 2: Run script
    • Step 3: Run script
    • Step 4: Run action
    ")
  end

  subgraph runner_2["Runner 2"]
    job_2("Job 2
    • Step 1: Run action
    • Step 2: Run script
    • Step 3: Run script
    ")
  end

:material-list-status: Workflow descriptions and status

“CI” - Core builds and tests

These workflows build the project from source then run unit tests.

  • To keep these workflows focused, they should not need any special hardware (e.g. GPUs).
  • Some workflows in this category use sanitizers, debug builds, alternate compilers, and other features that maintainers want automated coverage for.
Workflow fileBuild statusEvent triggers
ci.ymlcIpull_request, push
ci_linux_arm64_clang.ymlCI - Linux arm64 clangschedule
ci_macos_arm64_clang.ymlCI - macOS x64 clangschedule
ci_linux_x64_clang_byollvm.ymlCI - Linux x64 clang_byollvmschedule
ci_linux_x64_clang_debug.ymlCI - Linux x64 clang debugschedule
ci_linux_x64_clang_tsan.ymlCI - Linux x64 clang TSanschedule
ci_linux_x64_gcc.ymlCI - Linux x64 gccschedule

“PkgCI” - Package builds and tests

These workflows build packages from source then run test suites using them.

graph LR
  accTitle: Package tests
  accDescr {
    Package tests start with a build_package step.
    After build_package, individual jobs are run in parallel for NVIDIA t4
    tests, AMD mi300 tests, etc.
  }

  build_packages --> test_nvidia_t4
  build_packages --> test_amd_mi300
  build_packages --> test_etc
  • Treat test workflows as code that downstream users would write - the packages used in workflows should be interchangeable with packages installed directly from PyPI or GitHub Releases.
  • Test workflows can build the IREE runtime from source (possibly cross-compiling) but they should use iree-compile and any other host tools from the built packages.
  • Test workflows can install other packages (e.g. tensorflow, torch) and fetch from model repositories like Hugging Face as needed to run test suites.

!!! Tip

PkgCI workflows can be triggered directly to skip the 5-10 minute build
"build packages" job when testing workflows. See the
[faster iteration on PkgCI workflows](#faster-iteration-on-pkgci-workflows)
section below for details.
Workflow fileBuild statusEvent triggers
Package tests
pkgci.ymlPkgCIpull_request, push

Other workflows

Workflow fileBuild statusEvent triggers
build_package.ymlBuild Release Packagesschedule
publish_website.ymlpublish_websitepush, release, schedule
samples.ymlSamplesschedule

:octicons-pencil-16: Writing and editing workflows

:simple-docker: Docker and dependencies

Workflow files typically require some external dependencies in the form of software packages, environment settings, and sometimes even system/hardware drivers. One way to manage these dependencies is to bundle them into a container using a tool like Docker.

!!! tip

We recommend only using Docker containers within workflow files in specific
cicumstances and with moderation.

IREE contains a cross-compiler and minimal runtime, both of which are
designed to run on a wide range of systems. Using carefully constructed
containers for basic development risks the project _only_ working within
such containers.

These sorts of dependencies may be a good fit for using Docker containers:

  • Infrequently changing large dependencies like compiler toolchains.
  • Dependencies with complicated installs (e.g. building from source, moving files to specific paths).
  • System dependencies like GPU drivers.
  • Environment settings like disk partitions.

Here are alternative ways to fetch and configure workflow/job dependencies:

:octicons-workflow-16: Workflow triggers

Of the events that trigger workflows, we most commonly use:

  • pull_request
    • Jobs most representative of core developer workflows should aim to run here. Jobs can be marked required or opt-in on presubmit on a case-by-case basis.
  • push
    • Jobs running here should be a superset of jobs running on pull_request.
  • schedule
    • Jobs designed to run nightly (e.g. nightly releases), jobs for non-core configurations (like certain sanitizers/fuzzers), and jobs using self-hosted runners in low supply can run on a schedule instead of on every commit.
  • workflow_dispatch
    • This trigger is mostly used for manual workflow debugging.
    • Where possible, jobs should allow this trigger so maintainers can test workflows without needing to send pull requests.

!!! info - ““Presubmit” and “postsubmit””

We use the terminology "presubmit" and "postsubmit" to differentiate
between stages when checks run:

* "Presubmit" checks run on code that has not yet been
  reviewed/approved/merged with either of the `pull_request` or
  `workflow_dispatch` triggers.
* "Postsubmit" checks run on code that has been merged to a common branch
  like `main` with either of the `push` or `schedule` triggers.

In an ideal world every check would run on presubmit, but some operating
system or hardware runners are in short supply and some workflows are slow
even with sufficient resources (e.g. benchmark suites). We try to strike a
balance between utility and economics.

!!! example - “Example workflow triggers”

* [`ci_linux_x64_clang_asan.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_asan.yml)
  runs on presubmit (`pull_request` trigger) and postsubmit (`push`
  trigger). Even though this workflow builds the compiler and needs to use
  large build machines because, it is generally useful for all C/C++
  compiler and runtime changes.
* [`ci_linux_x64_clang_tsan.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_x64_clang_tsan.yml)
  is similar to the ASan build but it runs on the `schedule` event because
  it is only situationally useful and we want to limit use of large build
  machines. It would run on GitHub-hosted runners if they could handle it
  without running out of disk space.
* [`ci_linux_arm64_clang.yml`](https://github.com/iree-org/iree/blob/main/.github/workflows/ci_linux_arm64_clang.yml)
  uses the `schedule` event since GitHub does not offer free Linux arm64
  runners.

:octicons-check-circle-16: Required and optional checks

Any workflow that runs on the pull_request event can be either optional (the default) or required.

  • All required checks must be passing for a pull request to be merged.
  • Pull requests can be merged with optional checks pending or even failing.
  • The auto-merge feature will wait for required reviews to be met and required status checks to pass.

!!! note

Required checks must use only either standard GitHub-hosted runners or
runners from the CPU builder pool.

:octicons-skip-16: Opt-in for presubmit jobs

GitHub supports paths and paths-ignore filters for push and pull_request events that can be used to configure which workflows run based on paths modified. This mechanism is simple but somewhat limited in what it can express, so we have a custom mechanism for marking certain jobs as conditionally enabled:

  • Always run on push events, after pull requests are merged (postsubmit).

  • Jobs may be marked as opt-in for pull_request events (presubmit) by editing build_tools/github_actions/configure_ci.py. That script runs as part of the setup.yml action, which jobs can depend on like so:

    jobs:
  setup:
    uses: ./.github/workflows/setup.yml

  test:
    needs: [setup]
    if: contains(fromJson(needs.setup.outputs.enabled-jobs), 'test')
    steps:
      - ...

  • Opt-in jobs can also be set up to run automatically if specific file paths are modified, much like GitHub's paths and paths-ignore filters.

To bypass the computed configuration of workflows to run, see the CI behavior manipulation section of our contributing guide.

:octicons-server-16: Using GitHub-hosted and self-hosted runners

We group runners into categories:

  1. GitHub-hosted runners: standard (free)
    • Workflow jobs should use these GitHub-hosted runners when at all possible.
  2. GitHub-hosted runners: large (paid)
    • We aren't currently using these, but they have been useful where self-hosting is difficult.
  3. Self-hosted runners: CPU builders
    • The core project maintainers sponsor a pool of powerful CPU build machines used to build the core project and packages. To limit the load on this pool, we restrict jobs using these runners to only the most essential.
  4. Self-hosted runners: special hardware
    • Project contributors can offer their own hardware as self-hosted runners, allowing tests to be run on that hardware at whatever frequency they have the capacity to support.
    • Due to how difficult it can be to keep self-hosted runners operating reliably, and how access is limited to the group that maintains the hardware, any jobs using these self-hosted runners must be optional and easy to disable.
    • Self-hosted runners can either be ephemeral (one job per runner, compatible with autoscaling), or persistent. Persistent runners can retain local build and artifact caches to improve workflow time substantially.

!!! info - “Contributing self-hosted runners”

Want to run tests on your own hardware as part of IREE's upstream CI? Get
in touch with us on one of our
[communication channels](../../index.md#communication-channels) and we'd be
happy to discuss the options available.

Faster iteration on PkgCI workflows

Each pkgci workflow can be triggered directly with workflow_dispatch to run tests using a previous run of the build_packages job.

  1. Find the artifact_run_id from a prior run to use packages from. You can go through the history at https://github.com/iree-org/iree/actions/workflows/pkgci.yml for this. For example, https://github.com/iree-org/iree/actions/runs/13723791082 has run id 13723791082.

  2. Choose which workflow you want to run and navigate to its control page on https://github.com/iree-org/iree/actions, like https://github.com/iree-org/iree/actions/workflows/pkgci_unit_test.yml for pkgci_unit_test.yml.

  3. Run the workflow using the workflow_dispatch trigger, selecting the branch you want to test and providing the artifact_run_id from step 1:

    ci-extra

:material-book-open-outline: Maintenance tips

:material-help-box-multiple-outline: Useful resources

Official:

Community: