This directory contains the Dockerfiles that specify the container images used for IREE. Images are uploaded to Google Container Registry (GCR).
To build an image, use docker build, e.g.:
docker build build_tools/docker/cmake --tag cmake
To explore an image interactively, use docker run, e.g.
docker run --interactive --tty --rm cmake
You can find more information in the official Docker docs.
We use multi-stage builds to limit duplication in our Dockerfiles and reduce the final image size. There is still duplication in cases where it's difficult to determine the correct files to copy.
IREE images follow a consistent structure. The image defined by build_tools/docker/foo-bar/Dockerfile is uploaded to GCR as gcr.io/iree-oss/foo-bar. It may be tagged as latest or prod, e.g. gcr.io/iree-oss/foo-bar:latest. Dockerfile image definitions should list their dependencies based on these image names.
We use a helper python script to manage the Docker image deployment. It lists all images and their dependencies and manages their canonical registry location. When creating a new image, add it to this mapping. To build an image and all images it depends on:
python3 build_tools/docker/manage_images.py --build --image cmake
To build multiple images
python3 build_tools/docker/manage_images.py --build --image cmake --image bazel
There is also the special option --image all to build all registered images.
Pushing images to GCR requires the Storage Admin role in the iree-oss GCP project. To push these images to GCR with the latest tag:
python3 build_tools/docker/manage_images.py --image cmake --push
Kokoro build scripts and RBE configuration refer to images by their repository digest. You can update references to the digest:
python3 build_tools/docker/manage_images.py --images all --tag latest --update_references
This requires that the tagged image have a repository digest, which means it was pushed to or pulled from GCR.
If you have worked with the docker images before, it is prudent to follow the steps in the “Debugging” section below before continuing.
Update the Dockerfile for the image that you want to modify or add. If you're adding a new image, or updating the dependencies between images, be sure to update IMAGES_TO_DEPENDENCIES in manage_images.py as well.
Build the image, push the image to GCR and update all references to the image with the new GCR digest:
python3 build_tools/docker/manage_images.py \ --image "${IMAGE?}" --build \ --tag latest \ --push \ --update_references
Test that the changes behave as expected locally and iterate on the steps above.
Commit the changes and send a PR for review. The CI will use the updated digest references to test the new images.
Merge your PR after is approved and all CI tests pass. Please remember to complete the rest of the steps below.
:prod tagKokoro builds preload images tagged with prod on VM creation, so after changing the images used, you should also update the images tagged as prod in GCR. This also makes development significantly easier for others who need to modify the docker images.
On the main branch, build (but don't push) the images and locally tag them with the :prod tag:
python3 build_tools/docker/manage_images.py \ --image "${IMAGE?}" --build \ --tag prod \ --update_references
This build should be entirely cache hits.
We include --update_references in the command above so that we can check that none of the images or references to them have been changed. Check that the following command produces no output before continuing:
git status --porcelain
If the output is not empty then you‘ll need to find the source of the discrepancy (e.g. a locally modified Dockerfile) and remove it, and repeat steps 5 and 6 before continuing. (This relies on you keeping your local copy of the Docker images. If you didn’t, you'll have to manually pull the missing images by their digest).
Now that we've confirmed that none of the images were changed, we can push them to GCR with the :prod tag.
python3 build_tools/docker/manage_images.py \ --image "${IMAGE?}" \ --tag prod \ --push
Sometimes old versions of the :latest images can be stored locally and produce unexpected behaviors. The following commands will download all of the prod images and then update the images tagged with :latest on your machine (and on GCR).
# Pull all :prod images python3 build_tools/docker/manage_images.py --images all --pull --tag prod # Update the :latest images to match the :prod images. # If you have a clean workspace this _shouldn't_ require building anything as # everything should be cache hits from the :prod images downloaded above, but if # the :prod images are behind then that will not be the case and this may take # several hours (depending on your machine). python3 build_tools/docker/manage_images.py \ --images all --build \ --tag latest \ --push \ --update_references