| commit | 268b6de58fecfb3a9b314ee3baab771b82ea0488 | [log] [tgz] |
|---|---|---|
| author | Stella Laurenzo <stellaraccident@gmail.com> | Fri Jul 28 11:57:59 2023 -0700 |
| committer | GitHub <noreply@github.com> | Fri Jul 28 11:57:59 2023 -0700 |
| tree | d8a8f7e5bf204ab32b758a16852aa66712eda57e | |
| parent | 50873373b7dfcf88e2f7c21f42ad3135d626a8ac [diff] |
Rework consteval to be more memory efficient. (#14504) * Also adds an option to use the llvm-cpu backend instead of VMVX. This isn't quite ready for primetime because I need to isolate some flags better, but it can be used in a pinch. * Changes the way that JIT programs are constructed so that stateless functions are compiled at once for all constants and the data is passed in/out vs being cloned from module globals. * Each global has a public function to compute it, allowing us to run them one by one or in batches to keep memory use under control. * In this patch, I am just running them one by one and also haven't optimized memory by use of resources yet. Still a big improvement. * Drops compilation time of h2ogpt from ~45m to 2m47s. More optimization is possible. * JIT'ing globals now takes 52s for this model (vs ~42m with the Linalg const evaler). * Memory pressure is kept under control and does not regress from the current state (more optimization is possible - just a starting point). I also changed the behavior of the constexpr hoisting and jit passes to produce/process initializers with the `iree.compiler.consteval` attribute. This was advisable because in the new way of doing evaluation, we are doing a non-general transformation on the initializers and I didn't want it stumbling over arbitrary initializers in the wild. Having the behavior be opt-in seemed prudent. Flag changes: * `iree-consteval-jit-use-vmvx` (default true): Uses the VMVX backend. When false, uses the LLVMCPU backend. I'll be doing work to change this to false by default when LLVMCPU is available. * `iree-consteval-jit-debug`: Prints debugging information about constant evaluation. * `iree-opt-const-eval`: Flipped to true since it now only processes initializers targeted at it and is therefore safe to always have enabled.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.