commit 609767c77e248600349242e3fbf69bc9e0326940
author Geoffrey Martin-Noble <gcmn@google.com> Tue Nov 02 11:12:46 2021 -0700
committer GitHub <noreply@github.com> Tue Nov 02 11:12:46 2021 -0700
tree e8035505722a8b637b4c37196fdb642c6196248d
parent 8be94e44402758cc8ed942bcab291d25f283cf0b

Squash docker images (#7501)

Our previous image setup was trying to keep things separate in a defined
hierarchy and use multi-stage builds. This was for a few reasons:
1. It followed Docker's recommended
   [best practices](https://docs.docker.com/develop/develop-images/dockerfile_best-practices/).
2. It ensured that builds only had exactly the things we thought they
   needed and didn't grow any weird extra dependencies.
3. It separated concerns, so each image was doing a single specific
   thing.
4. It avoided monolithic architectures.
5. It aimed to keep images as small as possible for each transmission
   over the wire.

These were all great in theory. In practice, it meant we had 20 docker
images and it was confusing to keep track of them all. The major failing
here is that Dockerfile is just really not a composable format. The only
mechanism of composability is fetching specific files from another
image, which requires enumerating all such files: not easy for installs
that scatter files across the filesystem. This meant that we ended up
repeating ourselves anyway and duplicated expensive work. Dockerfile
commands that create layers in the image also don't have explicit
dependencies. If any layer underneath it is changed, the whole cache is
invalidated. This means rebuilds are far more frequent than you'd want
for this sort of architecture. Additionally, we've ended up at a place
where things like Python and its basic modules like numpy and yaml are
needed in pretty much every build, so trying to separate them out is
much less useful.

So instead, this PR squashes things down to 12 images: a base image that
has what we need for basic builds and all the non-conflicting
dependencies and child images that add specific conflicting toolchain
things. This makes all of the images that remain bigger because they
have stuff they absolutely don't need. In particular, the frontends
image now inherits from the android image, which I need for
https://github.com/google/iree/pull/7494. I don't really understand why
the Android NDK has to be so massive (over 4GB). Here's a
[comparison](https://gist.github.com/GMNGeoffrey/949ebb31748421c1922d1c874add46c5)
of the old and new images and their sizes. Note that some images weren't
shipped anywhere and were purely for intermediate images. The only such
remaining image is "frontends" which is specialized into swiftshader and
nvidia flavors.

Tested: Triggered all the non-default builds. I was going to do a
oneshot of the gradle build, but it looks like that workflow has
never been run?

![Build TFLite Android Library screenshot](https://user-images.githubusercontent.com/5732088/139757332-05d7f11c-eb8d-4cea-9183-d77c976859ab.png)