Add infrastructure for testing Colab notebooks and samples. (#6297)
Progress on https://github.com/google/iree/issues/6165
Test runs:
* https://buildkite.com/iree/scotttodd-test-pipeline/builds/29
* https://buildkite.com/iree/scotttodd-test-pipeline/builds/31
---
This enables us to test parts of the project that aren't directly integrated into our CMake or Bazel build systems. New scripts are added for freeform testing following the same commands or workflows that we recommend to users.
I'm planning on running these tests first on a schedule (post-submit). A few other configurations can be added over time:
* _after a nightly release is published_: just a change in triggering, no deeper integration
* _before a release is published_: integrate into the release pipeline, needs some python package rewiring
* _on presubmit_: possible for pure C API samples, harder for anything touching Python packages
---
I used Python's `venv` ([see docs](https://docs.python.org/3/library/venv.html)) from any scripts that interacts with pip packages so each sample is tested in isolation. That needed some extra setup in Docker, so I created a new 'samples' Docker image. That image should work with these scripts while running under non-root users (like our other pipelines), but it took some trial and error to find a setup I was happy with.
diff --git a/colab/tensorflow_hub_import.ipynb b/colab/tensorflow_hub_import.ipynb
index 9a5b7b3..9d755f6 100644
--- a/colab/tensorflow_hub_import.ipynb
+++ b/colab/tensorflow_hub_import.ipynb
@@ -411,9 +411,12 @@
"!cd {ARTIFACTS_DIR} && zip -r {ARTIFACTS_ZIP} .\n",
"\n",
"# Note: you can also download files using the file explorer on the left\n",
- "from google.colab import files\n",
- "print(\"Downloading the artifacts zip file...\")\n",
- "files.download(ARTIFACTS_ZIP)"
+ "try:\n",
+ " from google.colab import files\n",
+ " print(\"Downloading the artifacts zip file...\")\n",
+ " files.download(ARTIFACTS_ZIP)\n",
+ "except ImportError:\n",
+ " print(\"Missing google_colab Python package, can't download files\")"
],
"execution_count": 9,
"outputs": [
