Merge pull request #7998 from google/benvanik-resource-retain Command buffers that execute asynchronously now need to ensure the lifetime of all referenced resources extends for as long as the command buffer lives. Future changes will extend this to device queues/semaphores. By tracking lifetime on command buffers we can remove the deferred release list that was performing lifetime tracking on the HAL VM module - it was using a generic VM list (24 bytes per entry vs 8 on 64-bit) and limited us to a single device/command buffer and synchronous submission: now when we get around to attaching lifetime to timepoints we can correctly track lifetime in asynchronous programs 🎉 An important user-visible reason we bring the lifetime tracking into the HAL is that when we start to mix asynchronous host application code, compiler-generated code, and custom module code we have no good way of plumbing through the lifetimes with the current interface. A bonus of this change is that now we only pay for lifetime tracking when it's required: when running against implementations that perform their own resource lifetime tracking (WebGPU/Metal) we may be able to avoid doing the tracking. We can also avoid tracking when we are wrapping command buffers in things like the deferred command buffer (or future RPC systems) that perform the tracking themselves. This change drops our memory consumption high-water mark in dylib-sync/embedded cases by 16-32KB as now we don't need tracking there and improves the performance (~4us -> 3us per hello-world invocation) as there's no tracking logic taking place. It's better for the other backends that do require tracking (task system/GPUs/etc) due to reduced memory consumption, allocations, and a more optimized implementation.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.