blob: 718d53b11907a4f607d23931a2c90d899ee9bcda [file] [log] [blame] [view]
# Asynchronous tensor I/O custom module sample
This sample expects that you've already produced a working version of the
[basic sample](/samples/custom_module/basic/) (including compiler installation
and CMake setup).
This sample demonstrates adding custom modules callable from compiler-produced
programs that take and return `tensor` types. Both the calls into the compiled
program and the custom call made from the compiled program are made
asynchronously using HAL fences for ordering work. This allows the entire
invocation - including the custom user call - to be scheduled without blocking
and enables pipelining and overlapping invocations. When embedded into a larger
user-level framework this lets IREE invocations be interleaved with other user
work.
## Instructions
1. Compile the [example module](./test/example.mlir) to a .vmfb file:
```
iree-compile \
--iree-execution-model=async-external \
--iree-hal-target-backends=llvm-cpu \
samples/custom_module/async/test/example.mlir \
-o=/tmp/example.vmfb
```
2. Build the `iree_samples_custom_module_async_run` CMake target :
```
cmake -B ../iree-build/ -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo . \
-DCMAKE_C_FLAGS=-DIREE_VM_EXECUTION_TRACING_FORCE_ENABLE=1
cmake --build ../iree-build/ --target iree_samples_custom_module_async_run
```
(here we force runtime execution tracing for demonstration purposes)
[See here](https://iree.dev/building-from-source/getting-started/)
for general instructions on building using CMake.
3. Run the example program to call the main function:
```
../iree-build/samples/custom_module/async/custom-module-async-run \
/tmp/example.vmfb example.main
```
## TBD
* Expose a way to tie call arguments and results for in-place operations.
* Expose a way to specify the lifetime of the I/O to allow for transient memory.