| commit | 381a16c0972b2552e443116b041bdaad4c8896d1 | [log] [tgz] |
|---|---|---|
| author | Lei Zhang <antiagainst@gmail.com> | Tue Jan 09 17:47:58 2024 -0500 |
| committer | GitHub <noreply@github.com> | Tue Jan 09 14:47:58 2024 -0800 |
| tree | 0d7a14feaebece372edb067b33a1a90629d76f69 | |
| parent | a7a7ad61f5788e1f86b49831237ddc9a03a86342 [diff] |
[cuda] Fix deadlock when advancing deferred queue in driver thread (#15673) When GPU signals the CUevent-implemented semaphore to a new value, we need to perform a `cuLaunchHostFunc()` to invoke the CPU to signal to that value. At the same time, we can try to advance the deferred queue to release more workload to the GPU. Advancing the deferred queue cannot happen as part of the `cuLaunchHostFunc()` call, given that is in a driver thread. Per the documentation: The host function must not make any CUDA API calls. Attempting to use a CUDA API may result in CUDA_ERROR_NOT_PERMITTED, but this is not required. The host function must not perform any synchronization that may depend on outstanding CUDA work not mandated to run earlier. Host functions without a mandated order (such as in independent streams) execute in undefined order and may be serialized. We were actually doing that and seeing deadlock. So this commit changes the implemention to create a new thread dedicated to monitoring and releasing workload to the GPU, to avoid calling CUDA APIs as part of the `cuLaunchHostFunc()` call. Progress towards https://github.com/openxla/iree/issues/13245
IREE (Intermediate Representation Execution Environment, pronounced as “eerie”) is an MLIR-based end-to-end compiler and runtime that lowers Machine Learning (ML) models to a unified IR that scales up to meet the needs of the datacenter and down to satisfy the constraints and special considerations of mobile and edge deployments.
See our website for project details, user guides, and instructions on building from source.
IREE is still in its early phase. We have settled down on the overarching infrastructure and are actively improving various software components as well as project logistics. It is still quite far from ready for everyday use and is made available without any support at the moment. With that said, we welcome any kind of feedback on any communication channels!
See our website for more information.
IREE is licensed under the terms of the Apache 2.0 License with LLVM Exceptions. See LICENSE for more information.