samples/custom_module/async/README.md

# Asynchronous tensor I/O custom module sample

This sample expects that you've already produced a working version of the
[basic sample](/samples/custom_module/basic/) (including compiler installation
and CMake setup).

This sample demonstrates adding custom modules callable from compiler-produced
programs that take and return `tensor` types. Both the calls into the compiled
program and the custom call made from the compiled program are made
asynchronously using HAL fences for ordering work. This allows the entire
invocation - including the custom user call - to be scheduled without blocking
and enables pipelining and overlapping invocations. When embedded into a larger
user-level framework this lets IREE invocations be interleaved with other user
work.

## Instructions

1. Compile the [example module](./test/example.mlir) to a .vmfb file:

    ```
    iree-compile \
        --iree-execution-model=async-external \
        --iree-hal-target-backends=llvm-cpu \
        samples/custom_module/async/test/example.mlir \
        -o=/tmp/example.vmfb
    ```

2. Build the `iree_samples_custom_module_async_run` CMake target :

    ```
    cmake -B ../iree-build/ -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo . \
        -DCMAKE_C_FLAGS=-DIREE_VM_EXECUTION_TRACING_FORCE_ENABLE=1
    cmake --build ../iree-build/ --target iree_samples_custom_module_async_run
    ```
    (here we force runtime execution tracing for demonstration purposes)

    [See here](https://iree.dev/building-from-source/getting-started/)
    for general instructions on building using CMake.

3. Run the example program to call the main function:

   ```
   ../iree-build/samples/custom_module/async/custom-module-async-run \
       /tmp/example.vmfb example.main
   ```

## TBD

* Expose a way to tie call arguments and results for in-place operations.
* Expose a way to specify the lifetime of the I/O to allow for transient memory.