tree 25ea6c9c45ae70480fd42a5052a1f4a024c4a06e
parent d43294521aa2af71910d2a8a8fef56a7e5d36c91
author Phoenix Meadowlark <meadowlark@google.com> 1597866495 -0700
committer GitHub <noreply@github.com> 1597866495 -0700
gpgsig -----BEGIN PGP SIGNATURE-----
 
 wsBcBAABCAAQBQJfPYH/CRBK7hj4Ov3rIwAAdHIIAI0ukvXeKHcLbMyhKUjytjEH
 zdu8lVgHgP4jxjaeva3tL+JzqVMcUosUydiYLSCiwk9konufgLBCewdP2a+ifhtU
 kIuGoAI8SJV9DZix2Aoam65CJRd6QKk0CnweQHsIEobPEPQbYpFezJYDkTB3kpfU
 6Vj2+uzjdo/GZNYmoC8UMLzMqjroYzQDsg96Xd+c2mD1++84tB42BoQeE0Wlnx85
 q1PjImozFDhzqUPcdTugea1hafxllhXiMkdfNphhEioMEcwVcy/jPkkYd99SEnP7
 ksfv3I9fcOcssMbMQfxnnRyORn06NHN2IM6pMGqsl3lhweIKcewHRwsFLNXzCLM=
 =tWAn
 -----END PGP SIGNATURE-----
 

Add serialization for benchmarking e2e test modules with IREE (#2895)

Abseil flagfiles containing all of the data that `iree-benchmark-module` needs to run are generated for each `Trace` in our E2E tests. This allows for any module we test to be easily benchmarked on valid inputs. The process for benchmarking a vision model can thus be reduced to the following:

```shell
# Generate benchmarking artifacts for all vision models:
bazel test integrations/tensorflow/e2e/keras:vision_external_tests

# Benchmark ResNet50 with cifar10 weights on vmla:
bazel run iree/tools:iree-benchmark-module -- \
  --flagfile=/tmp/iree/modules/ResNet50/cifar10/iree_vmla/traces/predict/flagfile 

# Benchmark ResNet50 with cifar10 weights on llvmjit:
bazel run iree/tools:iree-benchmark-module -- \
  --flagfile=/tmp/iree/modules/ResNet50/cifar10/iree_llvmjit/traces/predict/flagfile 
```

Duplicate flags provided after the flagfile will take precedence. For example:

```shell
bazel run iree/tools:iree-benchmark-module -- \
  --flagfile=/tmp/iree/modules/ResNet50/cifar10/iree_llvmjit/traces/predict/flagfile  \
  --input_file=/path/to/custom/compiled.vmfb
```

Currently, this only supports benchmarking the first module call in a trace. We plan to extend this to support benchmarking all of the calls in the trace, and also plan to support verifying outputs during the warm-up phase of the benchmark.