This guide provides an overview of IREE's project structure and main tools for developers.
IREE‘s compiler components accept programs and code fragments in several formats, including high level TensorFlow Python code, serialized TensorFlow SavedModel programs, and lower level textual MLIR files using combinations of supported dialects like mhlo and IREE’s internal dialects. While input programs are ultimately compiled down to modules suitable for running on some combination of IREE‘s target deployment platforms, IREE’s developer tools can run individual compiler passes, translations, and other transformations step by step.
iree-opt is a tool for testing IREE‘s compiler passes. It is similar to mlir-opt and runs sets of IREE’s compiler passes on .mlir input files. See “conversion” in MLIR's Glossary for more information. Transformations performed by iree-opt can range from individual passes performing isolated manipulations to broad pipelines that encompass a sequence of steps.
Test .mlir files that are checked in typically include a RUN block at the top of the file that specifies which passes should be performed and if FileCheck should be used to test the generated output.
Here's an example of a small compiler pass running on a test file:
$ ../iree-build/iree/tools/iree-opt \ -split-input-file \ -mlir-print-ir-before-all \ -iree-drop-compiler-hints \ $PWD/iree/compiler/Dialect/Util/Transforms/test/drop_compiler_hints.mlir
For a more complex example, here‘s how to run IREE’s complete transformation pipeline targeting the VMVX backend on the fullyconnected.mlir model file:
$ ../iree-build/iree/tools/iree-opt \ -iree-transformation-pipeline \ -iree-hal-target-backends=vmvx \ $PWD/iree/test/e2e/models/fullyconnected.mlir
Custom passes may also be layered on top of iree-opt, see samples/iree_custom_modules/dialect for a sample.
iree-compile is IREE's main compiler driver for generating binaries from supported input MLIR assembly.
For example, to translate simple.mlir to an IREE module:
$ ../iree-build/iree/tools/iree-compile \ -iree-hal-target-backends=vmvx \ $PWD/samples/models/simple_abs.mlir \ -o /tmp/simple_abs_vmvx.vmfb
This is the IREE equivalent of MLIR's translation tool, which is used for testing translations between supported formats. It is used by various unit tests which are testing these features in isolation (outside of the main compiler driver).
See mlir-translate, see “translation” in MLIR's Glossary for more information.
Custom translations may also be layered on top of iree-translate, see samples/iree_custom_modules/dialect for a sample.
The iree-run-module program takes an already translated IREE module as input and executes an exported main function using the provided inputs.
This program can be used in sequence with iree-compile to translate a .mlir file to an IREE module and then execute it. Here is an example command that executes the simple simple_abs_vmvx.vmfb compiled from simple_abs.mlir above on IREE's VMVX driver:
$ ../iree-build/iree/tools/iree-run-module \ --module_file=/tmp/simple_abs_vmvx.vmfb \ --driver=vmvx \ --entry_function=abs \ --function_input=f32=-2
The iree-check-module program takes an already translated IREE module as input and executes it as a series of googletest tests. This is the test runner for the IREE check framework.
$ ../iree-build/iree/tools/iree-compile \ -iree-input-type=mhlo \ -iree-mlir-to-vm-bytecode-module \ -iree-hal-target-backends=vmvx \ $PWD/iree/test/e2e/xla_ops/abs.mlir \ -o /tmp/abs.vmfb
$ ../iree-build/iree/tools/iree-check-module \ /tmp/abs.vmfb \ --driver=vmvx
The iree-run-mlir program takes a .mlir file as input, translates it to an IREE bytecode module, and executes the module.
It is designed for testing and debugging, not production uses, and therefore does some additional work that usually must be explicit, like marking every function as exported by default and running all of them.
For example, to execute the contents of samples/models/simple_abs.mlir:
$ ../iree-build/iree/tools/iree-run-mlir \ $PWD/samples/models/simple_abs.mlir \ -function-input="f32=-2" \ -iree-hal-target-backends=vmvx
The iree-dump-module program prints the contents of an IREE module FlatBuffer file.
For example, to inspect the module translated above:
$ ../iree-build/iree/tools/iree-dump-module /tmp/simple_abs_vmvx.vmfb
There are a few useful generic flags when working with IREE tools:
--iree_minloglevel and --iree_vThese flags can control IREE tool output verbosity. --iree_minloglevel and --iree_v set the minimal and maximal verbosity levels respectively. They both accept a number where 0, 1, 2, 3 stands for info, warning, error, and fatal error respectively.
All the IREE tools support reading input values from a file. This is quite useful for debugging. Use -help for each tool to see what the flag to set. The inputs are expected to be newline-separated. Each input should be either a scalar or a buffer. Scalars should be in the format type=value and buffers should be in the format [shape]xtype=[value]. For example:
1x5xf32=1,-2,-3,4,-5 1x5x3x1xf32=15,14,13,12,11,10,9,8,7,6,5,4,3,2,1
iree-flow-trace-dispatch-tensorsThis flag will enable tracing inputs and outputs for each dispatch function. It is easier to narrow down test cases, since IREE breaks a ML workload into multiple dispatch function. When the flag is on, IREE will insert trace points before and after each dispatch function. The first trace op is for inputs, and the second trace op is for outputs. There will be two events for one dispatch function.
For IREE's Vulkan runtime driver, there are a few useful flags defined in driver_module.cc: