Asynchronous tensor I/O custom module sample

This sample expects that you've already produced a working version of the basic sample (including compiler installation and CMake setup).

This sample demonstrates adding custom modules callable from compiler-produced programs that take and return tensor types. Both the calls into the compiled program and the custom call made from the compiled program are made asynchronously using HAL fences for ordering work. This allows the entire invocation - including the custom user call - to be scheduled without blocking and enables pipelining and overlapping invocations. When embedded into a larger user-level framework this lets IREE invocations be interleaved with other user work.

Instructions

1. Compile the example module to a .vmfb file:

   iree-compile \
       --iree-execution-model=async-external \
       --iree-hal-target-backends=llvm-cpu \
       samples/custom_module/async/test/example.mlir \
       -o=/tmp/example.vmfb
   

2. Build the iree_samples_custom_module_async_run CMake target :

   cmake -B ../iree-build/ -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo . \
       -DCMAKE_C_FLAGS=-DIREE_VM_EXECUTION_TRACING_FORCE_ENABLE=1
   cmake --build ../iree-build/ --target iree_samples_custom_module_async_run
   

   (here we force runtime execution tracing for demonstration purposes)

   See here for general instructions on building using CMake.

3. Run the example program to call the main function:

   ../iree-build/samples/custom_module/async/custom-module-async-run \
       /tmp/example.vmfb example.main
   

TBD

• Expose a way to tie call arguments and results for in-place operations.
• Expose a way to specify the lifetime of the I/O to allow for transient memory.