)]}'
{
  "commit": "9ca7563d903d8e6f04e9f456f0cf1cf70546d506",
  "tree": "96ce003e60c7e83ccced9d4425d2ea9aae2e43ff",
  "parents": [
    "519815e4bcd51cf690ede07338d73d99241465b7"
  ],
  "author": {
    "name": "Scott Todd",
    "email": "scotttodd@google.com",
    "time": "Wed Apr 13 10:47:11 2022 -0700"
  },
  "committer": {
    "name": "GitHub",
    "email": "noreply@github.com",
    "time": "Wed Apr 13 10:47:11 2022 -0700"
  },
  "message": "Make benchmark artifact generation a bit more flexible. (#8846)\n\nHere\u0027s my use case:\r\n\r\n* I\u0027m working on Windows, where we don\u0027t support building the TensorFlow integration tools (notably `iree-import-tflite`)\r\n* I want to use the .mlir files that we feed into our benchmarking for local development (more specifically I want _some_ set of real programs, but the benchmark suite offers a set that we continuously build/test and have metrics for already)\r\n\r\n---\r\n\r\nNow, a sensible solution would be to extend\r\nhttps://github.com/google/iree/blob/1ddd9170bd6cc2701e2d669754a8fff248739233/build_tools/buildkite/cmake/android/arm64-v8a/benchmark2.yml#L9-L14\r\nto also zip up the imported .mlir files so anyone could download them from Buildkite.\r\n\r\n---\r\n\r\nI, however, had the more indirect idea to write a Colab notebook ([work in progress here](https://colab.research.google.com/gist/ScottTodd/4b27c6b44f90d239d4676dacddb87895/iree-benchmarks-import-demo.ipynb)) that installs the latest IREE release along with the project source and then builds `iree-benchmark-suites` using the installed tools. That _almost_ worked, except for the few issues that this PR patches over:\r\n\r\n* `$\u003cTARGET_FILE:iree::tools::iree-compile\u003e` only works when building with `IREE_BUILD_COMPILER`, but it\u0027s also useful to point at already built tools using `IREE_HOST_BINARY_ROOT`. The `iree_get_executable_path` function handles switching between those modes easily\r\n* Compiling imported files for the full list of benchmark configurations can be slow. If a developer just wants to import .mlir files then skipping those following steps would be useful. I added a new `iree-benchmark-import-models` target that does that.\r\n\r\n(I also considered the _even more indirect_ idea to parse the artifact paths from [benchmarks/TFLite/CMakeLists.txt](https://github.com/google/iree/blob/main/benchmarks/TFLite/CMakeLists.txt) and run `iree-import-tflite` manually, but the `iree-benchmark-suites` target already does that natively :P)\r\n\r\n---\r\n\r\nI\u0027m open to any workflow that solves my original use case, but these changes seem generally good to have anyways.",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "8a39a475887c2588d750bc59d37d5aa518566777",
      "old_mode": 33188,
      "old_path": "CMakeLists.txt",
      "new_id": "78109eb4321ff4ac487bd6b23fb1ca9843daeb8f",
      "new_mode": 33188,
      "new_path": "CMakeLists.txt"
    },
    {
      "type": "modify",
      "old_id": "a830c5e880afc13724e40a64053c315ec99a3d8d",
      "old_mode": 33188,
      "old_path": "build_tools/cmake/iree_benchmark_suite.cmake",
      "new_id": "34280a2baea0c808a4ab8fd5092fe969fbc6db16",
      "new_mode": 33188,
      "new_path": "build_tools/cmake/iree_benchmark_suite.cmake"
    }
  ]
}