| commit | ecc49f6a2a14d7edcd68340521e4b2d2500a3d8d | [log] [tgz] |
|---|---|---|
| author | Stella Laurenzo <stellaraccident@gmail.com> | Thu Aug 10 23:37:22 2023 -0700 |
| committer | GitHub <noreply@github.com> | Thu Aug 10 23:37:22 2023 -0700 |
| tree | 0cfcfa42b8aa11ea2b4884f9db10a93c93c45587 | |
| parent | 57b9239e0d4988dc440cbe794f689003714e2bda [diff] |
Add a CLI `iree-ir-tool` with a command to strip data. (#14636) Been meaning to do this for a while in order to have a place to stash more power-user style things that core developers typically use iree-opt for. This one adds a `strip-data` sub-command which uses the passes from #14627 to systematically replace tensor constants with synthetic values. With an ASAN/asserts build, this was able to strip a 7GiB int4 vicuna MLIR file in ~5s and a 23GiB h2ogpt model in about 30s (the latter has some other characteristics which make it more expensive to load as well as being bigger). Results were a 2.6MiB and 1.4MiB MLIR file respectively, consisting just of program IR and annotations for synthetic data. Getting the opt pipeline right for arbitrary input is a bit tricky, so I decided we should just armor this into a tool From installed packages, this can be used as: ``` iree-ir-tool strip-data input.mlir -o output.mlir ``` From a build tree with Python setup: ``` python -m iree.compiler.tools.ir_tool strip-data input.mlir -o output.mlir ``` Required adding some additional compiler APIs: * `ireeCompilerInvocationRunPassPipeline` to run an arbitrary textual pass pipeline on an invocation. * `ireeCompilerInvocationOutputIRBytecode` to emit bytecode from an invocation.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.