| commit | 5c576ca83c696e77ffa62bd8f782ed1eb5fd8605 | [log] [tgz] |
|---|---|---|
| author | Stanley Winata <68087699+raikonenfnu@users.noreply.github.com> | Fri Aug 23 09:42:08 2024 -0700 |
| committer | GitHub <noreply@github.com> | Fri Aug 23 09:42:08 2024 -0700 |
| tree | 515aa81e926ed75e50c00a0a2f2b31cfa973072e | |
| parent | a0945cc1aafbe2c9eabc807ddb6e8397ec6d0716 [diff] |
[Codegen][Common] Reorder layout analysis to preserve anchored to_layout (#18304) TL;DR: Re-ordered forward and backward layout. Added support to propagate through arith.select S.T setting to_layout at certain locations in decomposition gets honored. Cleaned up unused/old anchor maps. This change makes it easier to honor to_layout conversions generated by the users and/or higher level decompositons. Since intuitively to_layout op has the effect that ops that come after it shall take/be based on it's layout. Loading the forward propagation first, also encourage priortize ops to take layouts from anchors / to_layout op that comes before it. The motivation behind this PR was to solve the issue where despite us emitting to_layout op at certain locations, we ended up generating redundant to_simd earlier in the graph. This results us to not have control on where the layout conversion actually happens. One example of this case is when we emit a to_layout conversion for FP8 Attention. We emit a to_layout right before the 2nd contract and right after the truncf to FP8. Since we do enforcement(backward propagation) first and then propagate (forward propagation), we ended up generating two layout conversions, one the original one we emitted, the second one determined by layout analysis and placed in the middle of a elemwise op that is part of the softmax. This forces us to do shuffles/layout resolution in FP32 which hurts the performance. Additionally, this is also not where we(compiler writers) intended the layout conversion to happen. Additionally, we made modification to getAgreedLayout to handle elemwise propagation where some operands are not expected to have layout such as arith.select who's condition operand probably don't have layouts. --------- Signed-off-by: Stanley Winata <stanley.winata@amd.com>
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels
| Package | Release status |
|---|---|
| GitHub release (stable) |  |
| GitHub release (nightly) |  |
| Python iree-compiler |  |
| Python iree-runtime |  |
| Host platform | Build status |
|---|---|
| Linux |   |
| macOS |  |
| Windows |  |
For the full list of workflows see https://iree.dev/developers/general/github-actions/.
See our website for more information.
Community meeting recordings: IREE YouTube channel
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.