| commit | ae04c6701628095f54cddb7477fce346365d19e6 | [log] [tgz] |
|---|---|---|
| author | Quinn Dawkins <quinn.dawkins@gmail.com> | Fri Jun 07 15:50:53 2024 -0400 |
| committer | GitHub <noreply@github.com> | Fri Jun 07 12:50:53 2024 -0700 |
| tree | 244ff3a1af30aa156dfb678f7ab1066e2bc0ef19 | |
| parent | 3b5d269c7fec61743cc41f4394b33a31625ef2ae [diff] |
[Codegen][LLVMGPU] Add pass pipeline for greedy tile + fuse (#17559) This adds a pass pipeline based around recently added patterns to fuse and hoist scf.forall loops. The core idea of this pipeline is aimed at the requirements for matmul and convolution where they are tiled to a serial loop and then the tiles of the operands are copied first to a shared memory allocation, then loaded in the layout needed for the GEMM. This requires shared tiling for the reduction loop (tile + fuse), however tiling the input copies (or potential fused producers) requires different thread tiling for the matmul and the input operands. This pass pipeline addresses this issue by doing such tiling independently and then fusing the resulting loops together. The remaining TODOs for this pipeline are: 1. Distribution/fusion of consumers with the thread tiled matmul forall. 2. Adding a kernel config logic for the pipeline (matmul and conv). 3. Passes for packing matmuls to MMA intrinsic shapes and then tiling + distributing to intrinsics. This may require a tiling interface implementation for `iree_gpu.multi_mma`. 4. Fix an issue related to `iree_gpu.shuffle_tensor` and bufferization that currently is relying on the bufferization allocation function to pick the correct memory space. 5. Add img2col to this pass to allow doing convolutions with implicit gemm. 5a. Generalize the upstream img2col pass to enable more conv-like generics. 6. Add multibuffering/pipelining passes and make them configurable (ideally coming from attributes on lowering configs added to loops when tiling).
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
Community meeting recordings: IREE YouTube channel
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.