| commit | 27be42f696e5c6c98d31243cbcc8a169edb18a84 | [log] [tgz] |
|---|---|---|
| author | Ben Vanik <ben.vanik@gmail.com> | Mon Dec 05 19:25:32 2022 -0800 |
| committer | GitHub <noreply@github.com> | Mon Dec 05 19:25:32 2022 -0800 |
| tree | 5896d1a5cc340d0b212c7e893500003703578ff6 | |
| parent | be4c5fcb20834ed055b92e94fbf0d478ff012af2 [diff] | |
| parent | 0066c7a5f033d29be645845a7b58e95631bbf41e [diff] |
Adding collectives HAL operations and compiler support. (#11342) This adds end-to-end support from a new `stream.async.collective` op and communication channel type down through the HAL and all the way to runtime. Conceptually collective operations are commands that can be recorded into command buffers and the various HAL backends can decide how to implement them. The commands are transfer-like but may be implemented with dispatch logic. We could offer a local emulated channel that let us simulate multiple devices by performing copies but this first version just returns unimplemented on all backends. A skeleton of the runtime support is provided for NCCL support in the CUDA backend. In the future we can add local/ backend support for various collective libraries if we want or expose a factory mechanism on device creation to allow hosting applications control over the communication channels and routing. A utility has been added to allow command buffer implementations to accumulate batches of collective operations for efficient submission to the underlying library APIs. There are several areas future changes will focus on but what's here should be enough for some basic hello-world programs, major things missing: * send/recv are not available at the `stream.async.*` level * collectives cannot currently be performed in-place (#11249 is tracking the support required) * supported collective element types and reduction operators are basically just NCCL * emulation for unsupported element types and reduction operators is missing - we should insert casts and such at higher levels The next steps for wiring this up are to implement NCCL shared library loading, implement the `TODO(#9580)`s in the code for calling into NCCL, and some representation of collectives at the flow level that lower into the stream ops. Progress on #9580.
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.