| commit | b246fa621239bc593f1846e7102620113bf5eda2 | [log] [tgz] |
|---|---|---|
| author | Quinn Dawkins <quinn.dawkins@gmail.com> | Fri May 31 16:46:22 2024 -0400 |
| committer | GitHub <noreply@github.com> | Fri May 31 16:46:22 2024 -0400 |
| tree | a29301764fd23b4fad31d97ca44f5891fb50159e | |
| parent | 8e9b9a427a5cea3b18af336028e92dd5ca4e3fe1 [diff] |
[Codegen][GPU] Add pattern to distribute multi_mma ops to lanes (#17521) Distribution of an `iree_gpu.multi_mma` op does a one-shot rewrite from warp semantics to thread semantics, inserting a lane mapped scf.forall that extracts the appropriate slices from each operand according to the intrinsic kind. This also adds permutation fields to iree_gpu.multi_mma for representing permuted variants of the inner intrinsic matmul for subgroup semantics. By adding a permutation to the operands of the multi_mma operation, we can express row/column major inputs. and then distribute accordingly. There are two reasons for a generic "permutation" instead of a second set of indexing maps or something simpler like a boolean. 1) The permutations of each operand are unrelated, so indexing maps cannot relate two operands with one another. 2) Future batched intrinsics, and options to pre-swizzle/pack the inner dimensions can be managed with an implicit contract between the intrinsic kind and the permutation.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
Community meeting recordings: IREE YouTube channel
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.